-
Notifications
You must be signed in to change notification settings - Fork 3.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Hydra+DDP causing infinite hang #15545
Comments
@lminer is it possible to provide a config to test with and also provide the version of Hydra you are using? |
Is there a way to send it to you privately @jgbos |
@lminer You can try sending it to jgoodwin314@gmail.com if you want to do that. FYI, I have very limited time to debug though, so here are a couple things you could try out:
2 is essentially what lightning is doing under the hood (see here). If you get errors executing 2 there may be something wrong with your configs. |
@awaelchli I actually got it to work. It was indeed a problem with my configuration! |
Thank you @lminer for confirming! |
Bug description
I use hydra+ddp and upon installing 1.8 have found that any run I start with ddp, it now hangs with the following error:
I used git bisect and have traced the problem to this commit:
How to reproduce the bug
No response
Error messages and logs
Environment
More info
No response
cc @justusschock @kaushikb11 @awaelchli @akihironitta @rohitgr7
The text was updated successfully, but these errors were encountered: