New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
部署服务 #3648
Labels
solved
This problem has been already solved.
Comments
pip uninstall flash_attn |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Reminder
Reproduction
CUDA_VISIBLE_DEVICES=0,1 API_PORT=8000 llamafactory-cli api examples/inference/llama3_vllm.yaml
Expected behavior
CUDA_VISIBLE_DEVICES=0,1 API_PORT=8000 llamafactory-cli api examples/inference/llama3_vllm.yaml部署服务时报错ImportError: /usr/local/lib/python3.9/dist-packages/flash_attn_2_cuda.cpython-39-x86_64-linux-gnu.so: undefined symbol: _ZNK3c106SymIntltEl,请问设置什么参数可以不使用flash_attn呢?
System Info
No response
Others
No response
The text was updated successfully, but these errors were encountered: