New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Every run with CoreML "first run on a device may take a while ..." #2126
Comments
I'm playing with whisper.cpp today and was impressed by the result so I wanted to compare the large model and the large one using CoreML. 1 - I tried the same command 5 times in a row and every time I had the first time message appear:
Here are the results:
Maybe this is the correct behaviour and thus using the coreML version is not worth it on small samples or I'm simply doing something wrong. |
I'm on macOS 14.2.1 and I've run into the same problem. |
I'm on 14.4.1 and don't observe this issue - first time is slow, but next runs load fast |
The Metal backend was incorrectly initialized 2 times, which I support might have caused this problem. Should be fixed on latest |
M1 pro. os14.4.1. latest master pull and make with coreml:
--edit tried on macos 14.5 too. |
I'm on macOS 14.4.1, and testing out whisper.cpp.
I have compiled binaries with both CoreML and just Metal.
But every single time I run the CoreML version it spends an age loading the coreml model version, and doesn't seem to cache for the next run
Runs 1-3 with command
Not sure what I'm doing wrong, but I was expecting it to cache whatever it was doing so subsequent runs were quicker.
The text was updated successfully, but these errors were encountered: