Build Pip Wheels #523

AbdBarho · 2022-11-13T08:00:29Z

What does this PR do?

Adds Github Workflow to build pip wheels. Fixes #473.

Before submitting

Did you have fun?
- changing one line and then waiting a couple of hours to find out that I made a typo is not what I would define as fun, but I definitely learned a lot which is why I wanted to do this in the first place.
Did you read the contributor guideline?
Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- Pip installation fails, CUTCLASS not found #473
Did you make sure to update the docs?
- The xformers team should decide on the release and delivery processes of the pip wheels first, then the docs can be updated.
Did you write any new necessary tests?
- N/A
Did you update the changelog? (if needed)
- N/A

PR review

Anyone in the community is free to review the PR once the tests have passed.

AbdBarho · 2022-11-13T08:01:02Z

~~OOooops sorry! opened an MR in this repo instead of mine!~~

danthe3rd · 2022-11-13T08:13:28Z

Oh btw we would love to have official wheels in the repo, if you want to contribute a PR :)

AbdBarho · 2022-11-13T09:00:18Z

@danthe3rd Thanks for following up after my mistake!

I can gladly re-open this MR, it was intended as part of the discussion in #473 (comment).

However, there is a lot of stuff that is still unclear to me, maybe you or anyone else on the team can give me more input on this, just so I can know what to focus on:

The build is done in github workflows, but I see your ci infra is more towards circle ci, would this be a problem?
- I am using github workflows basically as "free cloud compute" since my 2015 laptop would probably melt before finishing installing nvcc
There is no automatic publishing of the wheels, instead, you get an artefacts.zip file at the end of the workflow which has to be downloaded / extracted / published manually (although this should be easy to fix, assuming the secrets are available)
I am currently building a wheel for each different python version, which seems like an absolute overkill, especially since the core of this repo is not python heavy, I am no pip/build expert, is there a way to build for all python versions as one package?
- Each wheel is roughly 200MB.
The build takes at least 1.5 hours per version (single threaded), with more CUDA architectures it takes even more, using ninja will get the runner killed for hugging resources. I am unsure if this can be optimized.
I have no idea of how to build / compile this repo on windows, I am also not sure if github provides windows runners in the non-enterprise version.
what to do with the "old" build scripts / infra ? would you prefer a github workflows only solution? or something more scripty?

Those are some of the questions that I have, the rest is just a consequence of the above.

Thank you! have a nice weekend.

danthe3rd · 2022-11-13T10:19:45Z

Let me get back to you next week. @fmassa @blefaudeux and @bottler might have some opinion as well

blefaudeux · 2022-11-14T20:35:56Z

Let me get back to you next week. @fmassa @blefaudeux and @bottler might have some opinion as well

pulling in @patrickvonplaten and @patil-suraj also, who were interested in helping or looking at that wheel building procedure at least

blefaudeux · 2022-11-14T20:42:58Z

Not personally in charge anymore, but trying to help out still :)

@danthe3rd Thanks for following up after my mistake!

I can gladly re-open this MR, it was intended as part of the discussion in #473 (comment).

However, there is a lot of stuff that is still unclear to me, maybe you or anyone else on the team can give me more input on this, just so I can know what to focus on:
* The build is done in github workflows, but I see your ci infra is more towards circle ci, would this be a problem?

I'm actually amazed that this works out on GHA, I didn´t think that was possible. I'm guessing that it will quickly count against the Meta credits, but it could be tied to a release (vs. for all PRs) and that would be probably good enough ?

  * I am using github workflows basically as "free cloud compute" since my 2015 laptop would probably melt before finishing installing `nvcc`

ahah, totally understood !

* There is no automatic publishing of the wheels, instead, you get an `artefacts.zip` file at the end of the workflow which has to be downloaded / extracted / published manually (although this should be easy to fix, assuming the secrets are available)

This can be automated indeed I believe, as you mentioned. I actually think that fairseq already has the flows for that

* I am currently building a wheel for each different python version, which seems like an absolute overkill, especially since the core of this repo is not python heavy, I am no `pip`/`build` expert, is there a way to build for all python versions as one package?

Not a real expert also, but should not be a blocker I think, these days it's trivial to change python versions (and most of them should be ABI compatible but I'm not 100% sure there)

  * Each wheel is roughly 200MB.

* The build takes at least 1.5 hours per version (single threaded), with more CUDA architectures it takes even more, using ninja will get the runner killed for hugging resources. I am unsure if this can be optimized.

* I have no idea of how to build / compile this repo on windows, I am also not sure if github provides windows runners in the non-enterprise version.

Not sure either but same as above, should not be a blocker I think, let's not make perfect the enemy of great :)

* what to do with the "old" build scripts / infra ? would you prefer a github workflows only solution? or something more [scripty](https://github.com/facebookresearch/xformers/blob/836768506d990fa3985dafd69a0b4518b3ae12bd/packaging/build_wheel.sh)?

as far as I'm concerned anything old can get the door if anything new is better 👯

Those are some of the questions that I have, the rest is just a consequence of the above.

Thank you! have a nice weekend.

Thank you, I didn´t know that this was up but it's great work, could be super duper useful to many people

danthe3rd

Actually the code in the PR looks good for a v1 - we can take care of the upload+windows later. Just added a few comments before we can merge it

.github/workflows/wheels.yml

danthe3rd · 2022-11-17T12:47:28Z

.github/workflows/wheels.yml

+      - name: Install nvcc
+        run: >
+          wget https://developer.download.nvidia.com/compute/cuda/repos/debian11/x86_64/cuda-keyring_1.0-1_all.deb &&
+          dpkg -i cuda-keyring_1.0-1_all.deb &&
+          apt-get update &&
+          apt-get install cuda-nvcc-11-8 cuda-libraries-dev-11-8 -y


we should install the version corresponding to matrix.cuda. There are some available github-action for that to help apparently: https://github.com/Jimver/cuda-toolkit

I am going to have to try this first.

~~Currently, the build happens in a debian slim container, which should (at least in theory) work with this action.~~ (I am still unsure)

PyTorch logs a warning to the console if the nvcc version is not the same as the one it was compiled with, but because it is only a minor version, pytorch logs that "this is likely not a problem" so I just winged it.

I ended up not using this action since it only works on debian but the container I am using is centos

.github/workflows/wheels.yml

facebook-github-bot · 2022-11-17T12:54:17Z

Hi @AbdBarho!

Thank you for your pull request and welcome to our community.

Action Required

In order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at cla@meta.com. Thanks!

patrickvonplaten · 2022-11-17T13:52:24Z

.github/workflows/wheels.yml

+  build_wheels:
+    strategy:
+      matrix:
+        # python: ["3.7", "3.8", "3.9", "3.10"]


Is pip currently not available for 3.7, 3.8, 3.9?

Quite the opposite, it is possible to compile to all of the above versions, this was just a test

patrickvonplaten · 2022-11-17T13:56:06Z

Hey @AbdBarho,

It's extremely cool that you're working on making pip wheels easily available for the community! I think many people use xformers fast attention mechanism and would love an easy way of installing it via pip.

As maintainers of the diffusers library: https://github.com/huggingface/diffusers we'd love to more prominently feature xformers for training and would be very happy to help you! We're also in contact with the PyTorch team (gently pinging @ngimel just FYI).

Please let us know if you're stuck with anything (github actions etc...) => we're happy to help!

AbdBarho · 2022-11-17T20:38:22Z

@patrickvonplaten Thank you very much!

I am going to try to fit in this in my schedule tomorrow, I will report back if I face any problems.

AbdBarho · 2022-11-18T22:00:27Z

This PR is now ready for review.

You can see the workflow and download the generated wheels here:
https://github.com/AbdBarho/xformers-wheels/actions/runs/3499213686

I ask you all please to test the built wheels in your applications, I did a test with python3.10, torch1.12.1, and cuda 11.6 on my SD docker, and it worked without a problem, the more tests the better.

danthe3rd

This looks great thank you! The CI failing is not due to this PR, so I'll merge.
Thanks a lot for your contribution!

@patrickvonplaten if you want to improve this workload, feel free to open a new PR.
In list of remaining things to do, I think we need to worry about windows as well (and as an extension, it would be great to have a CI for windows as well on all pull requests).

We don't plan to contribute to it in the short term, the only thing we will do is adding upload to pypi

susumuota · 2022-11-20T15:49:00Z

@AbdBarho
Hi, thanks you for the great PR!

I ask you all please to test the built wheels in your applications, I did a test with python3.10, torch1.12.1, and cuda 11.6 on my SD docker, and it worked without a problem, the more tests the better.

I have tested 3.10.6, 3.7.12 on GCE with Deep Learning VM and 3.7.15 at Google Colab.
There was no problem on 3.10.6 but I found a minor bug on 3.7s. Also, I fixed it and tested it on 3.7.12, 3.7.15 and 3.10.6.
I attached a Colab Notebook for details.

@danthe3rd
I got SyntaxError at /usr/local/lib/python3.7/dist-packages/xformers/ops/unbind.py, line 99

Python 3.7.15 (default, Oct 12 2022, 19:14:55) 
[GCC 7.5.0]
Commit hash: 47a44c7e421b98ca07e92dbf88769b04c9e28f86
Installing requirements for Web UI
Launching Web UI with arguments: --xformers --share
Cannot import xformers
Traceback (most recent call last):
  File "/content/stable-diffusion-webui/modules/sd_hijack_optimizations.py", line 18, in <module>
    import xformers.ops
  File "/usr/local/lib/python3.7/dist-packages/xformers/ops/__init__.py", line 19, in <module>
    from .swiglu_op import (  # noqa: F401
  File "/usr/local/lib/python3.7/dist-packages/xformers/ops/swiglu_op.py", line 14, in <module>
    from .unbind import stack_or_none, unbind
  File "/usr/local/lib/python3.7/dist-packages/xformers/ops/unbind.py", line 99
    return None, *grad.unbind(dim=ctx.dim)
                 ^
SyntaxError: invalid syntax

I tried to find info on xformers repository.

        return None, *grad.unbind(dim=ctx.dim)

Then I found this commit (#527) by @danthe3rd , and tried to fix like this,

        return None, *(grad.unbind(dim=ctx.dim))

But still it didn't work. Then, I found a info on Python 3.8's what's new page.

Generalized iterable unpacking in yield and return statements no longer requires enclosing parentheses.
so in Python 3.7, it requires enclosing parentheses at return statements!

It's a matter of return statement. So, this

        return None, *grad.unbind(dim=ctx.dim)

should be written like this in Python 3.7.

        return (None, *grad.unbind(dim=ctx.dim))

Here is a point of the difference between Python 3.7 and 3.8+.

$ python3.10
Python 3.10.6 | packaged by conda-forge | (main, Aug 22 2022, 20:36:39) [GCC 10.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> def test1():
...     a = (1, 2, 3)
...     return None, *a
...
>>> test1()
(None, 1, 2, 3)
>>> def test2():
...     a = (1, 2, 3)
...     return None, *(a)
...
>>> test2()
(None, 1, 2, 3)
>>> def test3():
...     a = (1, 2, 3)
...     return (None, *a)
...
>>> test3()
(None, 1, 2, 3)

$ python3.7
Python 3.7.12 | packaged by conda-forge | (default, Oct 26 2021, 06:08:21)
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> def test1():
...     a = (1, 2, 3)
...     return None, *a
  File "<stdin>", line 3
    return None, *a
                 ^
SyntaxError: invalid syntax
>>> def test2():
...     a = (1, 2, 3)
...     return None, *(a)
  File "<stdin>", line 3
    return None, *(a)
                 ^
SyntaxError: invalid syntax
>>> def test3():
...     a = (1, 2, 3)
...     return (None, *a)
...
>>> test3()
(None, 1, 2, 3)

Thanks!
xformers_py37_whl_fix.ipynb.zip

patrickvonplaten · 2022-11-21T10:37:39Z

Are pip wheels working now that this PR is merged?

AbdBarho closed this Nov 13, 2022

AbdBarho changed the title ~~update wheels~~ Build Pip Wheels Nov 13, 2022

danthe3rd mentioned this pull request Nov 17, 2022

Support for pip #532

Open

danthe3rd reviewed Nov 17, 2022

View reviewed changes

danthe3rd reopened this Nov 17, 2022

patrickvonplaten reviewed Nov 17, 2022

View reviewed changes

patrickvonplaten mentioned this pull request Nov 18, 2022

Meet error when enabling xformers on windows and doing loss backward huggingface/diffusers#1314

Closed

AbdBarho force-pushed the wheel branch from 6dea4b9 to 3a56341 Compare November 18, 2022 21:53

chore: build pip wheels

6e44cf1

AbdBarho force-pushed the wheel branch from 3a56341 to 6e44cf1 Compare November 18, 2022 21:54

danthe3rd approved these changes Nov 18, 2022

View reviewed changes

danthe3rd merged commit 103e863 into facebookresearch:main Nov 18, 2022

AbdBarho deleted the wheel branch November 19, 2022 06:28

This was referenced Nov 20, 2022

Making built wheels available for install #533

Closed

Build windows pip wheels #534

Merged

AbdBarho mentioned this pull request Dec 13, 2022

Publish Wheels to PyPi #588

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Build Pip Wheels #523

Build Pip Wheels #523

AbdBarho commented Nov 13, 2022 •

edited

Loading

AbdBarho commented Nov 13, 2022 •

edited

Loading

danthe3rd commented Nov 13, 2022

AbdBarho commented Nov 13, 2022 •

edited

Loading

danthe3rd commented Nov 13, 2022

blefaudeux commented Nov 14, 2022

blefaudeux commented Nov 14, 2022

danthe3rd left a comment

danthe3rd Nov 17, 2022

AbdBarho Nov 17, 2022 •

edited

Loading

AbdBarho Nov 18, 2022

facebook-github-bot commented Nov 17, 2022

patrickvonplaten Nov 17, 2022

AbdBarho Nov 17, 2022

patrickvonplaten commented Nov 17, 2022

AbdBarho commented Nov 17, 2022

AbdBarho commented Nov 18, 2022

danthe3rd left a comment

susumuota commented Nov 20, 2022

patrickvonplaten commented Nov 21, 2022

Build Pip Wheels #523

Build Pip Wheels #523

Conversation

AbdBarho commented Nov 13, 2022 • edited Loading

What does this PR do?

Before submitting

PR review

AbdBarho commented Nov 13, 2022 • edited Loading

danthe3rd commented Nov 13, 2022

AbdBarho commented Nov 13, 2022 • edited Loading

danthe3rd commented Nov 13, 2022

blefaudeux commented Nov 14, 2022

blefaudeux commented Nov 14, 2022

danthe3rd left a comment

Choose a reason for hiding this comment

danthe3rd Nov 17, 2022

Choose a reason for hiding this comment

AbdBarho Nov 17, 2022 • edited Loading

Choose a reason for hiding this comment

AbdBarho Nov 18, 2022

Choose a reason for hiding this comment

facebook-github-bot commented Nov 17, 2022

Action Required

Process

patrickvonplaten Nov 17, 2022

Choose a reason for hiding this comment

AbdBarho Nov 17, 2022

Choose a reason for hiding this comment

patrickvonplaten commented Nov 17, 2022

AbdBarho commented Nov 17, 2022

AbdBarho commented Nov 18, 2022

danthe3rd left a comment

Choose a reason for hiding this comment

susumuota commented Nov 20, 2022

patrickvonplaten commented Nov 21, 2022

AbdBarho commented Nov 13, 2022 •

edited

Loading

AbdBarho commented Nov 13, 2022 •

edited

Loading

AbdBarho commented Nov 13, 2022 •

edited

Loading

AbdBarho Nov 17, 2022 •

edited

Loading