Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

More robust FabricOptimizer/LightningOptimizer wrapping logic #18516

Merged
merged 5 commits into from
Sep 12, 2023
Merged

More robust FabricOptimizer/LightningOptimizer wrapping logic #18516

merged 5 commits into from
Sep 12, 2023

Conversation

awaelchli
Copy link
Contributor

@awaelchli awaelchli commented Sep 10, 2023 •
edited by github-actions bot
Loading

What does this PR do?

#18488 was a quick fix to keep the state saved in the __dict__ of the wrapper in sync with the wrapped optimizer. This PR proposes a different solution where we don't need to do that. There are no user facing changes and the tests added in #18488 show the refactor here retains the desired behavior.

📚 Documentation preview 📚: https://pytorch-lightning--18516.org.readthedocs.build/en/18516/

cc @Borda @justusschock @awaelchli @carmocca

@awaelchli awaelchli added refactor fabric lightning.fabric.Fabric optimizer fun Staff contributions outside working hours - to differentiate from the "community" label labels Sep 10, 2023
@awaelchli awaelchli marked this pull request as ready for review September 10, 2023 16:35
@github-actions
Copy link
Contributor

github-actions bot commented Sep 10, 2023 •
edited
Loading

⚡ Required checks status: All passing 🟢

Groups summary

🟢 pytorch_lightning: Tests workflow
Check ID Status
pl-cpu (macOS-11, lightning, 3.8, 1.11) success ✅
pl-cpu (macOS-11, lightning, 3.9, 1.12) success ✅
pl-cpu (macOS-11, lightning, 3.10, 1.13) success ✅
pl-cpu (macOS-11, lightning, 3.10, 2.0) success ✅
pl-cpu (macOS-11, lightning, 3.8, 1.11, oldest) success ✅
pl-cpu (ubuntu-20.04, lightning, 3.8, 1.11) success ✅
pl-cpu (ubuntu-20.04, lightning, 3.9, 1.12) success ✅
pl-cpu (ubuntu-20.04, lightning, 3.10, 1.13) success ✅
pl-cpu (ubuntu-20.04, lightning, 3.10, 2.0) success ✅
pl-cpu (ubuntu-20.04, lightning, 3.8, 1.11, oldest) success ✅
pl-cpu (windows-2022, lightning, 3.8, 1.11) success ✅
pl-cpu (windows-2022, lightning, 3.9, 1.12) success ✅
pl-cpu (windows-2022, lightning, 3.10, 1.13) success ✅
pl-cpu (windows-2022, lightning, 3.10, 2.0) success ✅
pl-cpu (windows-2022, lightning, 3.8, 1.11, oldest) success ✅
pl-cpu (macOS-11, pytorch, 3.8, 1.13) success ✅
pl-cpu (ubuntu-20.04, pytorch, 3.8, 1.13) success ✅
pl-cpu (windows-2022, pytorch, 3.8, 1.13) success ✅
pl-cpu (macOS-12, pytorch, 3.11, 2.0) success ✅
pl-cpu (ubuntu-22.04, pytorch, 3.11, 2.0) success ✅
pl-cpu (windows-2022, pytorch, 3.11, 2.0) success ✅

These checks are required after the changes to src/lightning/fabric/wrappers.py, src/lightning/pytorch/core/module.py, src/lightning/pytorch/core/optimizer.py, tests/tests_pytorch/core/test_lightning_optimizer.py.

🟢 pytorch_lightning: Azure GPU
Check ID Status
[pytorch-lightning (GPUs) (testing Lightning latest)](https://dev.azure.com/Lightning-AI/72ab7ed8-b00f-4b6e-b131-3388f7ffafa7/_build/results?buildId=174079&view=logs&jobId=47e66f3c-897a-5428-da11-bf5c7745762e) success
[pytorch-lightning (GPUs) (testing PyTorch latest)](https://dev.azure.com/Lightning-AI/72ab7ed8-b00f-4b6e-b131-3388f7ffafa7/_build/results?buildId=174079&view=logs&jobId=3f274fac-2e11-54ca-487e-194c91f3ae9f) success

These checks are required after the changes to src/lightning/pytorch/core/module.py, src/lightning/pytorch/core/optimizer.py, tests/tests_pytorch/core/test_lightning_optimizer.py, src/lightning/fabric/wrappers.py.

🟢 pytorch_lightning: Benchmarks
Check ID Status
lightning.Benchmarks success ✅

These checks are required after the changes to src/lightning/fabric/wrappers.py, src/lightning/pytorch/core/module.py, src/lightning/pytorch/core/optimizer.py.

🟢 fabric: Docs
Check ID Status
docs-make (fabric, doctest) success ✅
docs-make (fabric, html) success ✅

These checks are required after the changes to src/lightning/fabric/wrappers.py.

🟢 pytorch_lightning: Docs
Check ID Status
docs-make (pytorch, doctest) success ✅
docs-make (pytorch, html) success ✅

These checks are required after the changes to src/lightning/pytorch/core/module.py, src/lightning/pytorch/core/optimizer.py.

🟢 lightning_fabric: CPU workflow
Check ID Status
fabric-cpu (macOS-11, lightning, 3.8, 1.11) success ✅
fabric-cpu (macOS-11, lightning, 3.9, 1.12) success ✅
fabric-cpu (macOS-11, lightning, 3.10, 1.13) success ✅
fabric-cpu (macOS-11, lightning, 3.10, 2.0) success ✅
fabric-cpu (macOS-11, lightning, 3.8, 1.11, oldest) success ✅
fabric-cpu (ubuntu-20.04, lightning, 3.8, 1.11) success ✅
fabric-cpu (ubuntu-20.04, lightning, 3.9, 1.12) success ✅
fabric-cpu (ubuntu-20.04, lightning, 3.10, 1.13) success ✅
fabric-cpu (ubuntu-20.04, lightning, 3.10, 2.0) success ✅
fabric-cpu (ubuntu-20.04, lightning, 3.8, 1.11, oldest) success ✅
fabric-cpu (windows-2022, lightning, 3.8, 1.11) success ✅
fabric-cpu (windows-2022, lightning, 3.9, 1.12) success ✅
fabric-cpu (windows-2022, lightning, 3.10, 1.13) success ✅
fabric-cpu (windows-2022, lightning, 3.10, 2.0) success ✅
fabric-cpu (windows-2022, lightning, 3.8, 1.11, oldest) success ✅
fabric-cpu (macOS-11, fabric, 3.8, 1.13) success ✅
fabric-cpu (ubuntu-20.04, fabric, 3.8, 1.13) success ✅
fabric-cpu (windows-2022, fabric, 3.8, 1.13) success ✅
fabric-cpu (macOS-12, fabric, 3.11, 2.0) success ✅
fabric-cpu (ubuntu-22.04, fabric, 3.11, 2.0) success ✅
fabric-cpu (windows-2022, fabric, 3.11, 2.0) success ✅

These checks are required after the changes to src/lightning/fabric/wrappers.py, tests/tests_fabric/test_wrappers.py.

🟢 lightning_fabric: Azure GPU
Check ID Status
[lightning-fabric (GPUs) (testing Fabric latest)](https://dev.azure.com/Lightning-AI/72ab7ed8-b00f-4b6e-b131-3388f7ffafa7/_build/results?buildId=174081&view=logs&jobId=3f274fac-2e11-54ca-487e-194c91f3ae9f) success
[lightning-fabric (GPUs) (testing Lightning latest)](https://dev.azure.com/Lightning-AI/72ab7ed8-b00f-4b6e-b131-3388f7ffafa7/_build/results?buildId=174081&view=logs&jobId=47e66f3c-897a-5428-da11-bf5c7745762e) success

These checks are required after the changes to src/lightning/fabric/wrappers.py, tests/tests_fabric/test_wrappers.py.

🟢 mypy
Check ID Status
mypy success ✅

These checks are required after the changes to src/lightning/fabric/wrappers.py, src/lightning/pytorch/core/module.py, src/lightning/pytorch/core/optimizer.py.

🟢 install
Check ID Status
install-pkg (ubuntu-22.04, app, 3.8) success ✅
install-pkg (ubuntu-22.04, app, 3.11) success ✅
install-pkg (ubuntu-22.04, fabric, 3.8) success ✅
install-pkg (ubuntu-22.04, fabric, 3.11) success ✅
install-pkg (ubuntu-22.04, pytorch, 3.8) success ✅
install-pkg (ubuntu-22.04, pytorch, 3.11) success ✅
install-pkg (ubuntu-22.04, lightning, 3.8) success ✅
install-pkg (ubuntu-22.04, lightning, 3.11) success ✅
install-pkg (ubuntu-22.04, notset, 3.8) success ✅
install-pkg (ubuntu-22.04, notset, 3.11) success ✅
install-pkg (macOS-12, app, 3.8) success ✅
install-pkg (macOS-12, app, 3.11) success ✅
install-pkg (macOS-12, fabric, 3.8) success ✅
install-pkg (macOS-12, fabric, 3.11) success ✅
install-pkg (macOS-12, pytorch, 3.8) success ✅
install-pkg (macOS-12, pytorch, 3.11) success ✅
install-pkg (macOS-12, lightning, 3.8) success ✅
install-pkg (macOS-12, lightning, 3.11) success ✅
install-pkg (macOS-12, notset, 3.8) success ✅
install-pkg (macOS-12, notset, 3.11) success ✅
install-pkg (windows-2022, app, 3.8) success ✅
install-pkg (windows-2022, app, 3.11) success ✅
install-pkg (windows-2022, fabric, 3.8) success ✅
install-pkg (windows-2022, fabric, 3.11) success ✅
install-pkg (windows-2022, pytorch, 3.8) success ✅
install-pkg (windows-2022, pytorch, 3.11) success ✅
install-pkg (windows-2022, lightning, 3.8) success ✅
install-pkg (windows-2022, lightning, 3.11) success ✅
install-pkg (windows-2022, notset, 3.8) success ✅
install-pkg (windows-2022, notset, 3.11) success ✅

These checks are required after the changes to src/lightning/fabric/wrappers.py, src/lightning/pytorch/core/module.py, src/lightning/pytorch/core/optimizer.py.

Thank you for your contribution! 💜

Note
This comment is automatically generated and updates for 60 minutes every 180 seconds. If you have any other questions, contact carmocca for help.

carmocca
carmocca approved these changes Sep 10, 2023
View reviewed changes
Copy link
Contributor

@carmocca carmocca left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could do the same for the LightningOptimizer here too

@github-actions github-actions bot added the pl Generic label for PyTorch Lightning package label Sep 11, 2023
@awaelchli awaelchli changed the title More robust FabricOptimizer wrapping logic More robust FabricOptimizer/LightningOptimizer wrapping logic Sep 12, 2023
@Borda Borda merged commit e958c6f into master Sep 12, 2023
114 checks passed
@Borda Borda deleted the refactor/fabric-optimizer branch September 12, 2023 07:30
@mergify mergify bot added the ready PRs ready to be merged label Sep 12, 2023
@awaelchli awaelchli added this to the 2.1 milestone Sep 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Borda Borda Borda approved these changes

@carmocca carmocca carmocca approved these changes

@justusschock justusschock Awaiting requested review from justusschock justusschock is a code owner

@williamFalcon williamFalcon Awaiting requested review from williamFalcon

@tchaton tchaton Awaiting requested review from tchaton tchaton is a code owner

Assignees
No one assigned
Labels
fabric lightning.fabric.Fabric fun Staff contributions outside working hours - to differentiate from the "community" label optimizer pl Generic label for PyTorch Lightning package ready PRs ready to be merged refactor
Projects
None yet
Milestone
2.1
Development

Successfully merging this pull request may close these issues.
3 participants
@awaelchli @Borda @carmocca