RuntimeError: Expected B.stride(-1) == 1 to be true, but got false.

[AndyCao1125]

Dear authors,

Thanks for your work!

I'm facing some difficulties about install bimamba. I follow the instruction in the readme file with pip install -e causal-conv1d and pip install -e mamba and get results of causal-conv1d==1.0.0 and mamba-ssm==1.0.1.
With those packages, I can run the simple test.py sucessfully:

from mamba_ssm.modules.mamba_simple import Mamba    
mamba = Mamba(d_model=32,expand=2,bimamba=True).cuda()     
y = mamba(x) 

However, during training, the code returns error:

out = mamba_inner_fn_no_out_proj(
  File "/home/XXX/RL/VideoMamba/mamba/mamba_ssm/ops/selective_scan_interface.py", line 641, in mamba_inner_fn_no_out_proj
    return MambaInnerFnNoOutProj.apply(xz, conv1d_weight, conv1d_bias, x_proj_weight, delta_proj_weight,
  File "XXX/lib/python3.8/site-packages/torch/autograd/function.py", line 598, in apply
    return super().apply(*args, **kwargs)  # type: ignore[misc]
  File "XXX/lib/python3.8/site-packages/torch/cuda/amp/autocast_mode.py", line 115, in decorate_fwd
    return fwd(*args, **kwargs)
  File "/home/XXX/VideoMamba/mamba/mamba_ssm/ops/selective_scan_interface.py", line 222, in forward
    out, scan_intermediates, out_z = selective_scan_cuda.fwd(
RuntimeError: Expected B.stride(-1) == 1 to be true, but got false. (Could this error message be improved?  If so, please report an enhancement request to PyTorch.)

And I logged the B.stride(-1), it returns 1 at first, but suddenly returns 16 which was quite weird:(

Thus, I started searching the solutions. I found two similar issues focusing on this question: (1) in the Vim issue, one solution is to upgrate the causal-conv1d==1.1.0 (hustvl/Vim#41 (comment)); (2) Dr. Tao also said that this error is fixed in v1.1.0 (state-spaces/mamba#36 (comment)).

However, I tried many ways to upgrade the causal-conv1d, but all of them were failed.
My version is:

cuda 12.1
torch 2.3.1+cu121
Cython 0.29.32
python 3.8

(1) I simple use pip install causal-conv1d==1.1.0, it returns error:

RuntimeError: Error compiling objects for extension
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for causal-conv1d
  Running setup.py clean for causal-conv1d
Failed to build causal-conv1d
ERROR: Could not build wheels for causal-conv1d, which is required to install pyproject.toml-based projects

where this error may happen due to the incompatible version of many packages, e.g., cython.

(2) download the v1.1.0 version from its original github.

wget https://github.com/Dao-AILab/causal-conv1d/releases/download/v1.1.0/causal_conv1d-1.1.0+cu122torch2.2cxx11abiFALSE-cp38-cp38-linux_x86_64.whl
pip install causal_conv1d-1.1.0+cu122torch2.2cxx11abiFALSE-cp38-cp38-linux_x86_64.whl

But this casual-conv1d even cannot run the test.py successfully with error:

conv1d_out = causal_conv1d_cuda.causal_conv1d_fwd(x, conv1d_weight, conv1d_bias, True)
AttributeError: 'NoneType' object has no attribute 'causal_conv1d_fwd'

So I think this maually updated method is not preferable.

Hence, may I ask for your help? It's been bothering me for a while. Thank you so much!

[Andy1621]

I have not met this problem. And pip install causal-conv1dalso works for me. I'm not sure whether it's caused by the version oftorch`...

[AndyCao1125]

Maybe I should try some other versions... But thanks~
May I ask if the proper causal-conv1d version should be 1.1.0 rather than 1.0.0?

[Andy1621]

I think both is okay~

[AndyCao1125]

Thank!