Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bump NCCL floor to 2.18.1.1, relax PyTorch pin #218

Merged
merged 5 commits into from
Sep 25, 2024
Merged

bump NCCL floor to 2.18.1.1, relax PyTorch pin #218

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
bf7f169
try to reproduce nightly failure
jameslamb Sep 20, 2024
984dbd0
workflow dependencies
jameslamb Sep 20, 2024
3874227
bump nccl floor to 2.18.1.1
jameslamb Sep 20, 2024
9169d9e
revert testing-only changes
jameslamb Sep 20, 2024
ff12da5
relax pytorch pin
jameslamb Sep 23, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions conda/environments/all_cuda-118_arch-x86_64.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ dependencies:
- librmm==24.10.*,>=0.0.0a0
- nanobind>=0.2.0
- nbsphinx
- nccl
- nccl>=2.18.1.1
- ninja
- numpy>=1.23,<3.0a0
- numpydoc
Expand All @@ -40,7 +40,7 @@ dependencies:
- pytest-xdist
- python>=3.10,<3.13
- pytorch-cuda=11.8
- pytorch=2.0.0
- pytorch>=2.0,<2.4.0a0
- rapids-build-backend>=0.3.0,<0.4.0.dev0
- recommonmark
- scikit-build-core>=0.10.0
Expand Down
2 changes: 1 addition & 1 deletion conda/environments/all_cuda-125_arch-x86_64.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ dependencies:
- librmm==24.10.*,>=0.0.0a0
- nanobind>=0.2.0
- nbsphinx
- nccl
- nccl>=2.18.1.1
- ninja
- numpy>=1.23,<3.0a0
- numpydoc
Expand Down
2 changes: 1 addition & 1 deletion conda/recipes/libwholegraph/conda_build_config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ doxygen_version:
- ">=1.8.11"

nccl_version:
- ">=2.9.9"
- ">=2.18.1.1"

c_stdlib:
- sysroot
Expand Down
12 changes: 6 additions & 6 deletions dependencies.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ dependencies:
- libraft-headers==24.10.*,>=0.0.0a0
- librmm==24.10.*,>=0.0.0a0
- nanobind>=0.2.0
- nccl
- &nccl nccl>=2.18.1.1
specific:
- output_types: conda
matrices:
Expand Down Expand Up @@ -216,14 +216,14 @@ dependencies:
common:
- output_types: [conda]
packages:
- nccl
- *nccl
test_python:
common:
- output_types: [conda]
packages:
- c-compiler
- cxx-compiler
- nccl
- *nccl
- output_types: [conda, requirements]
packages:
- ninja
Expand Down Expand Up @@ -285,13 +285,13 @@ dependencies:
# If conda-forge supports the new cuda-* packages for CUDA 11.8
# at some point, then we can fully support/properly specify
# this environment.
- pytorch=2.0.0
- &pytorch pytorch>=2.0,<2.4.0a0
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

docs build here have been failing like this:

LibMambaUnsatisfiableError: Encountered problems while solving:
  - nothing provides _python_rc needed by python-3.12.0rc3-rc3_hab00c5b_1_cpython

Could not solve for environment specs
The following packages are incompatible
├─ cpuonly is requested and can be installed;
├─ python 3.12**  is installable with the potential options
│  ├─ python [3.12.0|3.12.1|...|3.12.5] would require
│  │  └─ python_abi 3.12.* *_cp312, which can be installed;
│  └─ python 3.12.0rc3 would require
│     └─ _python_rc, which does not exist (perhaps a missing channel);
└─ pytorch 2.0.0**  is not installable because there are no viable options
   ├─ pytorch 2.0.0 would require
   │  └─ python_abi 3.8.* *_cp38, which conflicts with any installable versions previously reported;
   ├─ pytorch 2.0.0 would require
   │  └─ python >=3.10,<3.11.0a0 , which conflicts with any installable versions previously reported;
   ├─ pytorch 2.0.0 would require
   │  └─ cpuonly <0 , which conflicts with any installable versions previously reported;
   ├─ pytorch 2.0.0 would require
   │  └─ python >=3.8,<3.9.0a0 , which conflicts with any installable versions previously reported;
   ├─ pytorch 2.0.0 would require
   │  └─ python >=3.9,<3.10.0a0 , which conflicts with any installable versions previously reported;
   ├─ pytorch 2.0.0 would require
   │  └─ python_abi 3.10.* *_cp310, which conflicts with any installable versions previously reported;
   ├─ pytorch 2.0.0 would require
   │  └─ python_abi 3.11.* *_cp311, which conflicts with any installable versions previously reported;
   ├─ pytorch 2.0.0 would require
   │  └─ python_abi 3.9.* *_cp39, which conflicts with any installable versions previously reported;
   └─ pytorch 2.0.0 would require
      └─ __cuda, which is missing on the system.

(build link)

because:

This PR proposes loosening the pin on pytorch here.

cugraph currently pins to pytorch>=2.0,<2.2.0a0 (code link), but might go to <2.4.0a0 in rapidsai/cugraph#4615 (comment).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree this seems sensible (especially for this release). In 24.12 we can look at relaxing this further

- pytorch-cuda=11.8
- matrix:
arch: aarch64
cuda: "11.8"
packages:
- pytorch=2.0.0
- *pytorch
- pytorch-cuda=11.8
- matrix:
packages:
Expand All @@ -318,7 +318,7 @@ dependencies:
common:
- output_types: [conda]
packages:
- pytorch=2.0.0
- *pytorch
- cpuonly
clang_tools:
common:
Expand Down