We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
I encountered this error when running the code:
ValueError: No available memory for the cache blocks. Try increasing gpu_memory_utilization when initializing the engine.
gpu_memory_utilization
But I have already made these settings
bonito = Bonito("BatsResearch/bonito-v1",gpu_memory_utilization=0.9)
, it seems like they are not working, what should I do?
The text was updated successfully, but these errors were encountered:
This is related to the issue in the vllm package (vllm-project/vllm#2248).
You could try the following with Bonito and see if that helps:
bonito = Bonito("BatsResearch/bonito-v1", enforce_eager=True)
Sorry, something went wrong.
No branches or pull requests
I encountered this error when running the code:
ValueError: No available memory for the cache blocks. Try increasing
gpu_memory_utilization
when initializing the engine.But I have already made these settings
bonito = Bonito("BatsResearch/bonito-v1",gpu_memory_utilization=0.9)
, it seems like they are not working, what should I do?
The text was updated successfully, but these errors were encountered: