New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Supports cpu without avx2 instruction set #1142
Labels
enhancement
New feature or request
Comments
This was referenced Dec 29, 2023
wsxiaoys
changed the title
Supports avx2 in tabby
Supports cpu without avx2 instruction set
Dec 29, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Starting from version 0.5.0, the switch to llama.cpp in inference means that, by default, Tabby is compiled with AVX2 support. Consequently, Tabby may not function properly on CPUs lacking AVX2 support.
To address this issue, we could explore potential improvements by offering a separately compiled binary or awaiting an upstream llama.cpp patch that enables dynamic dispatching.
The text was updated successfully, but these errors were encountered: