-
-
Notifications
You must be signed in to change notification settings - Fork 440
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
404 when trying Qwen in V3 #723
Comments
|
Hello! Just a beginner here, could someone help me demonstrate with example code how to override the behaviour yourself using the model_file_name option when loading the model |
// using pipeline function
let pipe = await pipeline('text-generation', 'Xenova/Qwen1.5-0.5B-Chat', {model_file_name: 'decoder_model_merged'})
// using AutoModel class
let model = await AutoModel.from_pretrained('Xenova/Qwen1.5-0.5B-Chat', {model_file_name:'decoder_model_merged'})
// will fetch decoder_model_merged_quantized.onnx |
Question
This is probably just because V3 is a work in progress, but I wanted to make sure.
When trying to run Qwen 1.5 - 0.5B it works with the V2 script, but when swapping to V3 I get a 404 not found.
It seems V3 is looking for a file that was renamed 3 months ago.
Rename onnx/model_quantized.onnx to onnx/decoder_model_merged_quantized.onnx
I've tried setting
dtype
to 16 and 32, which does change the URL it tries to get, but those URL's also do not exist :-De.g.
https://huggingface.co/Xenova/Qwen1.5-0.5B-Chat/resolve/main/onnx/model_fp16.onnx
when usingdtype: 'fp16'
.Is there something I can do to make V3 find the correct files?
(I'm still trying to find that elusive small model with a large context size to do document summarization with)
The text was updated successfully, but these errors were encountered: