Skip to content

Commit

Permalink
refreshed /docs
Browse files Browse the repository at this point in the history
  • Loading branch information
@finale80
finale80 committed Oct 21, 2023
1 parent 79c3ebb commit 9399439
Show file tree
Hide file tree
Showing 4 changed files with 72 additions and 3 deletions.
31 changes: 29 additions & 2 deletions docs/papers/imc23/artifacts/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1736,8 +1736,10 @@ <h2 id="downloading-artifacts"><span class="twemoji"><svg xmlns="http://www.w3.o
</code></pre></div></p>
</li>
<li>
<p>Clone the tcbench repo with the imc23 tag
<div class="highlight"><pre><span></span><code><a id="__codelineno-1-1" name="__codelineno-1-1" href="#__codelineno-1-1"></a>
<p>Clone the tcbench repo and use the <code>imc23</code> branch
<div class="highlight"><pre><span></span><code><a id="__codelineno-1-1" name="__codelineno-1-1" href="#__codelineno-1-1"></a>git clone https://github.com/tcbenchstack/tcbench.git tcbench.git
<a id="__codelineno-1-2" name="__codelineno-1-2" href="#__codelineno-1-2"></a>cd tcbench.git
<a id="__codelineno-1-3" name="__codelineno-1-3" href="#__codelineno-1-3"></a>git checkout imc23
</code></pre></div></p>
</li>
<li>
Expand All @@ -1751,6 +1753,31 @@ <h2 id="downloading-artifacts"><span class="twemoji"><svg xmlns="http://www.w3.o
</code></pre></div></p>
</li>
</ul>
<p>This will install locally</p>
<ul>
<li>
<p>The <a href="/papers/imc23/notebooks/">notebooks</a> for replicating tables and figures of the paper under <code>/notebooks/imc23</code>.
The cloned repository already contains the notebooks but since the code might
change, the version fetched from figshare is identical to what used for the submission.</p>
</li>
<li>
<p>The <a href="/papers/imc23/ml_artifacts/">ml-artifacts</a> under <code>/notebooks/imc23/campaigns</code>.</p>
</li>
<li>
<p>The <a href="/papers/imc23/pytest/">pytest resources</a> for enabling unit tests.</p>
</li>
</ul>
<div class="admonition warning">
<p class="admonition-title">Packages depencency version and <code>/imc23</code> branch</p>
<p>When installing tcbench via pypi of from the main branch of the repository,
only a few sensible packages have a pinned version.</p>
<p>If you are trying to replicate the results of the paper, please
refer to the <code>/imc23</code> branch which also contains a
<code>requirements-imc23.txt</code> generated via <code>pip freeze</code> from
the environment used for collecting results. </p>
<p>Based on our experience, the most probable cause of results inconsistency
is due to package version. </p>
</div>



Expand Down
42 changes: 42 additions & 0 deletions docs/papers/imc23/pytest/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1600,6 +1600,48 @@


<h1 id="ml-unit-testing">ML unit testing<a class="headerlink" href="#ml-unit-testing" title="Permanent link">&para;</a></h1>
<p>Multiple tests are available to verify different functionalities
for either tcbench and the modeling campaigns created.</p>
<p>Tests are not bundled with pypi installation. Rather, you need
to follow the procedure described in the <a href="/papers/imc23/artifacts/">artifact page</a>
to fetch the source code and install all artifacts and datasets.</p>
<p>Tests are coded via <a href="https://docs.pytest.org/en/7.4.x/"><code>pytest</code> <span class="twemoji"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M2.615 0v.887h3.84V0zm5.022 0v.887h3.841V0zm4.957 0v.887h3.841V0zm4.935 0v.887h3.842V0zM2.447 1.895a.935.935 0 0 0-.935.935c0 .517.418.938.935.938h19.106c.517 0 .935-.42.935-.938a.935.935 0 0 0-.935-.936zm.168 2.847V24h3.84V4.742zm5.022 0v15.801h3.842v-15.8zm4.957 0v10.549h3.85V4.742zm4.935 0v6.494h3.842V4.742z"/></svg></span></a>
and are available under the <code>/tests</code> folder.</p>
<div class="admonition warning">
<p class="admonition-title">Tests trigger model training</p>
<p>Most of the test verify that the models train for
the campaigns described in the paper are indeed reproducible, i.e.,
the provide the exact same models obtained for the paper.</p>
<p>To do so, the pytest resources fetched from figshare
contains a subset of reference models so the test
trigger the modeling for those scenarios and check
that what trained matches what created for the paper.</p>
<p>So be aware that running these tests might take a while
depending on your local environment.</p>
</div>
<p>To trigger all tests run</p>
<div class="highlight"><pre><span></span><code><a id="__codelineno-0-1" name="__codelineno-0-1" href="#__codelineno-0-1"></a>pytest tests
</code></pre></div>
<div class="admonition note">
<p class="admonition-title">Output</p>
<div class="highlight"><pre><span></span><code><a id="__codelineno-1-1" name="__codelineno-1-1" href="#__codelineno-1-1"></a>============================ test session starts ======================================
<a id="__codelineno-1-2" name="__codelineno-1-2" href="#__codelineno-1-2"></a>platform linux -- Python 3.10.13, pytest-7.4.2, pluggy-1.3.0
<a id="__codelineno-1-3" name="__codelineno-1-3" href="#__codelineno-1-3"></a>rootdir: /tmp/tcbench-pip/tcbench
<a id="__codelineno-1-4" name="__codelineno-1-4" href="#__codelineno-1-4"></a>plugins: anyio-3.7.1, helpers-namespace-2021.12.29
<a id="__codelineno-1-5" name="__codelineno-1-5" href="#__codelineno-1-5"></a>collected 101 items
<a id="__codelineno-1-6" name="__codelineno-1-6" href="#__codelineno-1-6"></a>
<a id="__codelineno-1-7" name="__codelineno-1-7" href="#__codelineno-1-7"></a>tests/test_augmentations_at_loading.py ........... [ 10%]
<a id="__codelineno-1-8" name="__codelineno-1-8" href="#__codelineno-1-8"></a>tests/test_augmentations_at_loading_xgboost.py . [ 11%]
<a id="__codelineno-1-9" name="__codelineno-1-9" href="#__codelineno-1-9"></a>tests/test_cli_command_campaign.py .... [ 15%]
<a id="__codelineno-1-10" name="__codelineno-1-10" href="#__codelineno-1-10"></a>tests/test_cli_command_singlerun.py ............ [ 27%]
<a id="__codelineno-1-11" name="__codelineno-1-11" href="#__codelineno-1-11"></a>tests/test_contrastive_learning_and_finetune.py .. [ 29%]
<a id="__codelineno-1-12" name="__codelineno-1-12" href="#__codelineno-1-12"></a>tests/test_libtcdatasets_datasets_utils.py ................. [ 46%]
<a id="__codelineno-1-13" name="__codelineno-1-13" href="#__codelineno-1-13"></a>tests/test_modeling_backbone.py ................ [ 62%]
<a id="__codelineno-1-14" name="__codelineno-1-14" href="#__codelineno-1-14"></a>tests/test_modeling_dataprep.py .................................. [ 96%]
<a id="__codelineno-1-15" name="__codelineno-1-15" href="#__codelineno-1-15"></a>tests/test_modeling_methods.py .... [100%]
<a id="__codelineno-1-16" name="__codelineno-1-16" href="#__codelineno-1-16"></a>============================== 101 passed, 8 warnings in 6523.55s (1:48:43) =========================
</code></pre></div>
</div>



Expand Down
2 changes: 1 addition & 1 deletion docs/search/search_index.json
View file

Large diffs are not rendered by default.

Binary file modified docs/sitemap.xml.gz
View file
Binary file not shown.

0 comments on commit 9399439

Please sign in to comment.