-
Notifications
You must be signed in to change notification settings - Fork 686
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
try to run gemma-7b but failed #327
Comments
I have the same problem. It seems like the
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Recently, I integrated webllm into my web project, and the effect of gemma-2b is pretty good. Thanks for your work, everything runs very well. I am trying to add more powerful models.
I noticed that there is a quantized version of gemma-7b on huggingface but there are no related libs in https://github.com/mlc-ai/binary-mlc-llm-libs.
I tried to compile the wasm of gemma-7b according to the documentation, and then found that there was an error when loading the model to 8?/101
I saw "Here" in the console, and followed the code to find it here.
https://github.com/apache/tvm/blob/657880cdcedd7e41e911c583a8e93b3053a6ad27/web/src/runtime.ts#L82
Here is my configuration
I successfully compiled
gemma-7b-it-q4f16_2-metal.so
following the instructions here: https://github.com/mlc-ai/notebooks/blob/main/mlc-llm/models/demo_gemma.ipynb. It runs successfully on my MBP. It seems that the bug only appears in wasm.Has anyone successfully run gemma-7b? or any suggestions on how to troubleshoot this issue?
The text was updated successfully, but these errors were encountered: