-
Notifications
You must be signed in to change notification settings - Fork 68
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Phi-2 on iOS, bad responses #37
Comments
I noticed mlx-community/phi-2-hf-4bit-mlx has some custom python code which should be ran when using the python version of MLX. Since this custom code does not run on swift, does the Phi class in LLMEval implement a port of that custom code? |
No, the code has some simple prompt augmentation, but it doesn't use the config: For the general issue of debugging the tokenizer, since we have a working python version it is probably easiest to compare to that. I wrote up some thoughts here, see if they help: |
I am running the phi-2 on iOS using the code from LLMEval.
I have ported over an implementation of the CodeGen Tokenizer into swift as a standalone file:
I have now having an issue generating the response from phi-2. Even though the tokenizer seems to be correct, the response from Phi-2 is incoherent.
For reference here is a snippet of the code which generates tokens:
Here are the print logs:
Can anyone point me in the right direction to fixing this?
The text was updated successfully, but these errors were encountered: