[BUG] Unable to download CLIP model for search #4117
Comments
dankasak
added
bug
|
Hello, is this a clean instance, or this instance had been successfully using the search mechanism before? Please try to remove the volume for |
|
Hi! Thanks for responding.
Yes - all I've done so far is a batch import via the cli, and had a quick play.
No, it's never worked.
I backed up the model-cache directory, then created a fresh one, and restarted everything. It still fails, however I see the HTTP error code has changed from:
I've tried with curl, and I can see that curl now concurs - I'm now not able to download this file manually. I still have the one I previously downloaded. I also tried commenting out the model-cache volume stuff - same error as above. |
|
hey there! RuntimeError: Failed to download https://clip-as-service.s3.us-east-2.amazonaws.com/models-436c69702d61732d53657276696365/onnx/ViT-B-32/textual.onnx within retry limit 3 |
|
sadly, same problem for me. open the link https://clip-as-service.s3.us-east-2.amazonaws.com/models-436c69702d61732d53657276696365/onnx/ViT-B-32/textual.onnx in the Edge. It responds that: 
I tested in the mainland of China. The problem may be relevant to the GFW or it is simply because amazonaws blocks all Chinese IPs. |
|
I have exactly the same problem on a new install |
|
imho this is not an immich issue, looks like there is a problem with clip-as-service |
|
Having this issue as well, does not appear to be geographic. |
|
Unfortunately same issue for me with smart search. Metadata search (i.e., prefix with 'm:' still works as expected). Relevant immich-machinelearning logs (not to look like a broken record):
|
|
As this issue still exists and seems to affect more and more people, because new installs can't download the clip model, there should be implemented some sort of fallback to counter such situations in the future. Maybe a fallback model could be distributed which would spring into action if no new (/updated) model can be downloaded? Because I think there is no sense in a (local and) aspecially vital function like a search to fail, because of some external service not being available or something. |
|
@mertalev do you have any thought ok this issue? |
|
Hi there, |
|
Hmm, it might be better to use the models provided by Marqo instead since they're hosted in HF. This would have faster download speeds as well. |
|
@mertalev can we change that to make the model as default? |
|
I can look at it later today. Marqo uses a different naming scheme, so I'd need to map it to the same cache folder names we use to avoid duplicating models, and also migrate the model name in the system config. It shouldn't be too much work outside of that. |
|
Is there any workaround for those who don't have the file in cache? |
|
Thank you for the quick response - but I had tried using the Marqo models, but loading the model causes an error in the ML component. Not sure if it's a different bug? |
|
Oh, that's interesting. Looking at the text model's graph, I think it expects the attention mask to be pre-applied so there's only one input. Looks like I'll need to change the preprocessing. Jina:
Marqo:
|
|
For the use-case of Immich app packages (in our case for Cloudron), would it make sense to fetch and include those files during package/image building? If so, is there some pre-fetch command available somewhere to do this or a common place to check the URLs and versions for the models which should be fetched? |
|
PSA - the Jina AI textual.onnx bucket is working again, as is smart search on Immich. Suspect there will still be good enhancements that come out of this - thank you community! |
|
Sorry, what is the final fix for these issue? I tried to copy this files now I got this error. |
|
Since the S3 bucket now works, we haven't needed to make any changes for the time-being. I'm assuming you came to this issue because you're having problems downloading the CLIP model. Can you delete your |
|
Thank you @mertalev, after deleting the volume the search now works. I guess I have to run the "ENCODE CLIP" job again? |
|
You can run a "missing" job for Encode CLIP, but no need to run it on all images. |
|
Thank you @mertalev. It works :) |
|
Hello! |
|
Failed to download https://clip-as-service.s3.us-east-2.amazonaws.com/models-436c69702d61732d53657276696365/onnx/ViT-H-14-laion2b-s32b-b79k/textual.onnx with <HTTPError 403: 'Forbidden'> at the 0th |
|
Firstly, just want to say, I installed Immich for the first time this weekend and, wow. The software is amazing and the community is even better! So much support and documentation. I was also having the issue of not being able to download the models. What solved it for me was: sudo su
cd /var/lib/docker/volumes/immich_model-cache/_data/clip/ViT-B-32__openai
wget http://95.216.206.130/clip/ViT-B-32__openai/textual.onnx
wget http://95.216.206.130/clip/ViT-B-32__openai/visual.onnx |
|
The manual fix also worked for me! thank you very much. I would just like to point out that this functionality is still broken on the Immich demo page. |
|
thanks for that! am i right in assuming that i need to re-run "ENCODE CLIP" and "TAG OBJECTS" in the jobs page? |
|
Hi @FelixBuehler, Yes, I re-ran "ENCODE CLIP" and search started working for me again. In my case, it was a new instance of Immich, so that was the first time ENCODE CLIP actually ran at all. |
alextran1502
changed the title
|
@KjeldsenDK I am on windows + docker desktop as well. The way I added it to my docker volume model cache:
|
|
All links have been fixed and working again |
|
@NiklasRosenstein hello Does this model have it microsoft/resnet-50 |
|
@alextran1502 hi microsoft/resnet-50 model Do you have resources |
Can't download this model? |
It seems that the download link is down. Due to network issue I have to download every model manually. But I don't know the file structure of model-cache foldr and naming rule of model in the folder. Would anyone offer a file structure of model-cache folder? I would appreciate it very much. |
If you are having network issues while downloading I would recommend you to use a free VPN like Proton to bypass the limit temporarily If that not work for you I can send you the |
i've searched multiple issues and coomments and this seems to be a feasable fix for my problem, but the link is down. |
Thanks for your advice. To permanently solve the problem I managed to build a tproxy on the server. |
|
i checked my log and I'm having trouble downloading all the models needed by machine learning [11/18/23 11:03:16] INFO Initialized request thread pool with 8 threads. I don't know if it is a connection issue or what, seems like the program failed to even create the folders to save those files, including not only buffalo_l but also the ones mentioned in the above comments. I thought manually put those files in the cache folder might help but did not work, probably because I put them in the wrong place? |
What happens in terms of permissions? Do you have permissions to access the folder? |
I don't know how to check permissions under docker, I opened a new issue for details of yml files, please check: thanks for the help, I'm new to linux and still learning |
and others
Important
🟢 See this comment for temporary solution 🟢
The bug
When I search for anything in immich, I get generic errors in the UI. In docker logs, I can see that something is trying to download Downloading clip model 'ViT-B-32::openai' ... and "This may take a while". However it fails within about 3 seconds. I've downloaded this on the host using curl. Can I persist this somewhere for whatever needs it ... and if so, where? Why is it failing so quickly?
This seems to be triggered from:
https://github.com/jina-ai/clip-as-service/blob/main/server/clip_server/model/clip_onnx.py
The OS that Immich Server is running on
Docker
Version of Immich Server
v1.78.0
Version of Immich Mobile App
v1.78.0
Platform with the issue
Your docker-compose.yml content
Your .env content
Reproduction steps
Additional information
A search in the UI will trigger a download of https://github.com/jina-ai/clip-as-service/blob/main/server/clip_server/model/clip_onnx.py which will fail almost immediately
The text was updated successfully, but these errors were encountered: