CI: Test against old versions of key dependencies #16570

seberg · 2024-08-15T13:30:40Z

This adds explicit tests with old versions of key dependencies. Specifically:

numba==0.57
numpy==1.23
pandas==2.0
~~fsspec==0.6.0~~ excluded it. transformers==4.39.3 requires huggingface_hub which requires fsspec>=2023.5.0. In principle one could include it e.g. only for conda which doesn't pull in transformers, but that seemed not worth the trouble?
cupy==12.0.0
pyarrow==16.1.0

(Marking as draft until I see that things work.)

This adds explicit tests with old versions of key dependencies. Specifically: - `numba==0.57` - `numpy==1.23` - `pandas==2.0` - `fsspec==0.6.0` - `cupy==12.0.0` - `pyarrow==16.1.0`

copy-pr-bot · 2024-08-16T11:43:40Z

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

huggingface is pulled in for some jobs, which in turns pulls in fsspec but with a higher version constraint of 2023.5.0. So while the old fsspec is probably fine as a direct dependency we cannot (generally) enforce it in all tests. Signed-off-by: Sebastian Berg <sebastianb@nvidia.com>

seberg · 2024-08-16T13:25:41Z

@jameslamb maybe you can have a look w.r.t. to the setup/yaml? One of the builds currently failed, but I doubt a bit that it is related to the change (because build steps should be unaffected).

@mroeschke pinging maybe you know well what to do with the failures already showing up? Many seem just simple but e.g. this run has a surprising amount of failures.

(Have to check that this doesn't accidentally drop running "latest" tests for any setup.)

EDIT: Hmmm, I guess #15100 might be a good start for things that need to be readded if running <2.2.

ci/test_python_common.sh

jameslamb · 2024-08-19T16:10:04Z

@jameslamb maybe you can have a look w.r.t. to the setup/yaml? One of the builds currently failed, but I doubt a bit that it is related to the change (because build steps should be unaffected).

At a quick glance, changes look like what I'd expect. Nice trick including cupy== alongside cupy-cu{major}x together! I agree with you, that's a nice advantage of --constraint over --requirement.

If you fix conflicts + apply the one small suggestion I left, hopefully you'll see builds pass here. Once you've done that, @ me and I'd be happy to review again and more carefully.

Co-authored-by: James Lamb <jaylamb20@gmail.com>

jameslamb · 2024-08-19T21:41:24Z

@seberg I just merged #16575 which is going to create even more merge conflicts for you, sorry 😬

Could you try to follow the patterns from that PR? They're a little bit stricter and safer than what we had before in the case where a CI job is installing multiple other CI artifacts (e.g. cudf_polars jobs installing cudf, pylibcudf, and cudf_polars wheels).

mroeschke · 2024-08-21T01:45:35Z

pinging maybe you know well what to do with the failures already showing up?

From a glance the flavors of errors appears to be:

A potential bug/behavior that existed in pandas 2.0 that is different on pandas 2.2.x (which cudf tries to align with)
A new API that was introduced post pandas 2.0 (I recently made cudf APIs align with pandas 2.2.x)

IIRC from offline discussion of testing multiple pandas versions that we didn't want to overly complicate the test suite with version checking due to discrepancies in minor versions. I suspect we can largely use this pytest mark pattern for these:

@pytest.mark.skipif(
    PANDAS_VERSION < PANDAS_CURRENT_SUPPORTED_VERSION,
    reason="Fails in older versions of pandas",
)

dependencies.yaml

vyasr · 2024-08-28T21:16:28Z

ci/cudf_pandas_scripts/pandas-tests/run.sh

+if [[ $RAPIDS_DEPENDENCIES == "oldest" ]]; then
+    # `test_python` constraints are for `[test]` not `[cudf-pandas-tests]`
+    rapids-dependency-file-generator \
+        --output requirements \
+        --file-key test_python \
+        --matrix "cuda=${RAPIDS_CUDA_VERSION%.*};arch=$(arch);py=${RAPIDS_PY_VERSION};dependencies=${RAPIDS_DEPENDENCIES}" \
+    | tee ./constraints.txt
+fi


Do we think this is going to be a common pattern that's worth extracting into a gha-tool generate-oldest-constraints $file_key or similar? @jameslamb WDYT? It occurs many times in this PR and I assume we'll do something similar in other repos.

Yeah I think that's worth doing!

The only part that will vary from repo-to-repo is --file-key, everything else here is mechanical and a great candidate for a gha-tools script. I could put one up in a day or two, unless you or @seberg want to do it.

Would suggest one of the two of you pick this up. Sebastian is out for a bit

If you could tackle this that would be great @jameslamb. No rush. @mroeschke if you get everything else here finalized and have it ready to merge, I'm fine merging this PR with this code in and refactoring in a follow-up when James has the gha-tool ready.

Put up a proposal in rapidsai/gha-tools#114

ci/test_wheel_cudf.sh

vyasr · 2024-08-28T21:23:17Z

python/cudf/cudf/tests/test_join_order.py

+
+else:
+
+    def expect_inner(left, right, sort):


Could you explain which tests you've chosen to xfail and which ones are skipped in this PR? AFAIK since all the not PANDAS_GE_220 checks are negated, all the conditions are applying to tests that failed in older versions of pandas but pass on current pandas, which seems like they would be just as OK to skip. My comment above may have been unclear; to me, xfailing is most useful when we anticipate eventually getting an xpass as a signal. In these cases I don't think we ever will because we're xfailing tests on old versions that will only pass on newer versions. I'm fine using a different rationale if you think it's appropriate, but that's the source of my confusion here.

Co-authored-by: Vyas Ramasubramani <vyas.ramasubramani@gmail.com>

…t-oldest

bdice

I'm fine with all the changes here! Thanks team.

charlesbluca

Approving dask-cuDF code changes

mroeschke · 2024-09-03T22:49:42Z

Looks like with this new configuration, any ad-hoc job using test_python_common dependency will use the oldest dependencies by default e.g unit-tests-cudf-pandas, pandas-tests, wheel-tests-cudf-polars, wheel-tests-cudf-dask

Shouldn't we use the latest dependencies by default?

Specifically, the pandas-tests job clone the pandas unit tests from the version 2.2 tests but this job installs pandas 2.0, so there may be some incompatibility that's causing this job to time out

vyasr · 2024-09-04T00:55:25Z

The oldest dependency usage is entirely governed by the logic in the various ci/* scripts, which are reading the value of RAPIDS_DEPENDENCIES and setting up a constraint accordingly. If there are jobs that should not respect that and should always run with the latest deps (and you're right, I think there are) then those scripts should remove that logic entirely.

…t-oldest

mroeschke · 2024-09-04T17:53:55Z

Do other have thoughts whether the wheel-tests-cudf-polars or wheel-tests-cudf-dask jobs should just run with the oldest dependencies or should they run with the latest dependencies?

vyasr · 2024-09-04T22:32:36Z

I think it's sufficient for just the one job running the main cudf tests to be run with oldest dependencies for now. We'll probably want to do something similar for polars once we start supporting a range. For dask, I don't think we'll ever be able to support a range anyway; we always pin tightly.

mroeschke · 2024-09-04T23:11:02Z

/merge

This adds explicit tests with old versions of key dependencies. Specifically: - `numba==0.57` - `numpy==1.23` - `pandas==2.0` - ~`fsspec==0.6.0`~ excluded it. `transformers==4.39.3` requires `huggingface_hub` which requires `fsspec>=2023.5.0`. In principle one could include it e.g. only for conda which doesn't pull in `transformers`, but that seemed not worth the trouble? - `cupy==12.0.0` - `pyarrow==16.1.0` See also rapidsai/build-planning#81 (Marking as draft until I see that things work.) Authors: - Sebastian Berg (https://github.com/seberg) - Matthew Roeschke (https://github.com/mroeschke) - GALI PREM SAGAR (https://github.com/galipremsagar) Approvers: - Matthew Roeschke (https://github.com/mroeschke) - Bradley Dice (https://github.com/bdice) - Vyas Ramasubramani (https://github.com/vyasr) - Charles Blackmon-Luca (https://github.com/charlesbluca) URL: rapidsai#16570

seberg added improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Aug 15, 2024

seberg mentioned this pull request Aug 15, 2024

Testing against oldest python dependencies rapidsai/build-planning#81

Open

14 tasks

CI: Test against old versions of key dependencies

749752d

This adds explicit tests with old versions of key dependencies. Specifically: - `numba==0.57` - `numpy==1.23` - `pandas==2.0` - `fsspec==0.6.0` - `cupy==12.0.0` - `pyarrow==16.1.0`

seberg force-pushed the test-oldest branch 2 times, most recently from 5af6f41 to 8edb468 Compare August 16, 2024 11:14

Use constraints for pip (incidentally fixing cupy install issues)

c4f8547

seberg force-pushed the test-oldest branch from 8edb468 to c4f8547 Compare August 16, 2024 11:34

seberg force-pushed the test-oldest branch from 9c01a85 to 3a416aa Compare August 16, 2024 11:44

Consolidate pip installs, fix constraints and use for cudf-pandas tests

b22d111

seberg added 2 commits August 16, 2024 15:41

TST: xfail test that clearly requires newer pandas versions

224f944

TST: Reinstate (most) test version checks removed in 15100

b926cd3

github-actions bot added Python Affects Python cuDF API. cudf.pandas Issues specific to cudf.pandas labels Aug 16, 2024

jameslamb reviewed Aug 19, 2024

View reviewed changes

ci/test_python_common.sh Outdated Show resolved Hide resolved

seberg and others added 4 commits August 19, 2024 18:40

Update ci/test_python_common.sh

1d6c037

Co-authored-by: James Lamb <jaylamb20@gmail.com>

Merge branch 'branch-24.10' into test-oldest

e97d6fe

Clean up pylibcudf merge conflict

d84b0e3

Merge branch 'branch-24.10' into test-oldest

b5c17a2

bdice mentioned this pull request Aug 20, 2024

Enable testing cudf.pandas unit tests for all minor versions of pandas #16595

Merged

3 tasks

seberg added 4 commits August 21, 2024 12:38

Merge branch 'branch-24.10' into test-oldest

c267b1e

Use oldest deps also in cudf_pandas_scripts/pandas-tests/run.sh

062f601

TST: More (heavy handed) test fixups to ignore old pandas failures

4fff8b5

More heavy handed test fixes

3420a8c

mroeschke added 2 commits August 23, 2024 12:16

Merge remote-tracking branch 'upstream/branch-24.10' into test-oldest

f740c67

Check for min numba

273e18d

vyasr reviewed Aug 28, 2024

View reviewed changes

dependencies.yaml Outdated Show resolved Hide resolved

mroeschke added 2 commits August 28, 2024 13:56

Merge remote-tracking branch 'upstream/branch-24.10' into test-oldest

b8fcbe7

Make pyarrow pin 14 instead of 16

3a709a0

vyasr reviewed Aug 28, 2024

View reviewed changes

mroeschke and others added 6 commits August 28, 2024 14:26

Update ci/test_wheel_cudf.sh

5c794ae

Co-authored-by: Vyas Ramasubramani <vyas.ramasubramani@gmail.com>

Merge remote-tracking branch 'upstream/branch-24.10' into test-oldest

c7452fb

Change some xfails to skip

d09c47f

Merge branch 'test-oldest' of https://github.com/seberg/cudf into tes…

f9eb3ab

…t-oldest

Merge remote-tracking branch 'upstream/branch-24.10' into test-oldest

769f37b

skip test_p2p_shuffle for min pyarrow version

d701297

bdice approved these changes Aug 29, 2024

View reviewed changes

vyasr approved these changes Aug 29, 2024

View reviewed changes

galipremsagar and others added 2 commits August 29, 2024 22:48

Merge branch 'branch-24.10' into test-oldest

a994001

Merge branch 'branch-24.10' into test-oldest

309dd5b

charlesbluca approved these changes Sep 3, 2024

View reviewed changes

mroeschke added 2 commits September 3, 2024 08:44

Merge branch 'branch-24.10' into test-oldest

54b646e

Merge remote-tracking branch 'upstream/branch-24.10' into test-oldest

3928ba1

mroeschke added 2 commits September 3, 2024 18:37

Remove oldest dependency version checking from pandas unit test script

e34a02c

Merge branch 'test-oldest' of https://github.com/seberg/cudf into tes…

d157f1e

…t-oldest

rapids-bot bot merged commit ad1369d into rapidsai:branch-24.10 Sep 4, 2024
88 checks passed

jameslamb mentioned this pull request Sep 5, 2024

introduce rapids-generate-pip-constraints rapidsai/gha-tools#114

Open

seberg deleted the test-oldest branch September 10, 2024 16:49

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CI: Test against old versions of key dependencies #16570

CI: Test against old versions of key dependencies #16570

seberg commented Aug 15, 2024 •

edited

Loading

copy-pr-bot bot commented Aug 16, 2024

seberg commented Aug 16, 2024 •

edited

Loading

jameslamb commented Aug 19, 2024

jameslamb commented Aug 19, 2024

mroeschke commented Aug 21, 2024

vyasr Aug 28, 2024

jameslamb Aug 28, 2024

jakirkham Aug 29, 2024

vyasr Aug 29, 2024

jameslamb Sep 5, 2024

vyasr Aug 28, 2024

bdice left a comment

charlesbluca left a comment

mroeschke commented Sep 3, 2024

vyasr commented Sep 4, 2024

mroeschke commented Sep 4, 2024 •

edited

Loading

vyasr commented Sep 4, 2024 •

edited

Loading

mroeschke commented Sep 4, 2024

CI: Test against old versions of key dependencies #16570

CI: Test against old versions of key dependencies #16570

Conversation

seberg commented Aug 15, 2024 • edited Loading

copy-pr-bot bot commented Aug 16, 2024

seberg commented Aug 16, 2024 • edited Loading

jameslamb commented Aug 19, 2024

jameslamb commented Aug 19, 2024

mroeschke commented Aug 21, 2024

vyasr Aug 28, 2024

Choose a reason for hiding this comment

jameslamb Aug 28, 2024

Choose a reason for hiding this comment

jakirkham Aug 29, 2024

Choose a reason for hiding this comment

vyasr Aug 29, 2024

Choose a reason for hiding this comment

jameslamb Sep 5, 2024

Choose a reason for hiding this comment

vyasr Aug 28, 2024

Choose a reason for hiding this comment

bdice left a comment

Choose a reason for hiding this comment

charlesbluca left a comment

Choose a reason for hiding this comment

mroeschke commented Sep 3, 2024

vyasr commented Sep 4, 2024

mroeschke commented Sep 4, 2024 • edited Loading

vyasr commented Sep 4, 2024 • edited Loading

mroeschke commented Sep 4, 2024

seberg commented Aug 15, 2024 •

edited

Loading

seberg commented Aug 16, 2024 •

edited

Loading

mroeschke commented Sep 4, 2024 •

edited

Loading

vyasr commented Sep 4, 2024 •

edited

Loading