-
Notifications
You must be signed in to change notification settings - Fork 43.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(forge/llm): Add LlamafileProvider
#7091
base: master
Are you sure you want to change the base?
feat(forge/llm): Add LlamafileProvider
#7091
Commits on Apr 18, 2024
-
Add minimal implementation of LlamafileProvider, a new ChatModelProvi…
…der for llamafiles. Currently it just extends OpenAIProvider and only overrides methods that are necessary to get the system to work at a basic level. Update ModelProviderName schema and config/configurator so that app startup using this provider is handled correctly. Add 'mistral-7b-instruct-v0' to OpenAIModelName/OPEN_AI_CHAT_MODELS registries.
Configuration menu - View commit details
-
Copy full SHA for 03d8e1e - Browse repository at this point
Copy the full SHA 03d8e1eView commit details -
Adapt model prompt message roles to be compatible with the Mistral-7b…
…-Instruct chat template, which supports the 'user' & 'assistant' roles but does not support the 'system' role.
Configuration menu - View commit details
-
Copy full SHA for ed1dfd0 - Browse repository at this point
Copy the full SHA ed1dfd0View commit details -
In
OpenAIProvider
, change methodscount_message_tokens
, `count_to……kens`, and `get_tokenizer` from classmethods so I can override them in LlamafileProvide (and so I can access instance instance attributes from inside them). Implement class `LlamafileTokenizer` that calls the llamafile server's `/tokenize` API endpoint.
Configuration menu - View commit details
-
Copy full SHA for c56c290 - Browse repository at this point
Copy the full SHA c56c290View commit details -
Configuration menu - View commit details
-
Copy full SHA for 234d059 - Browse repository at this point
Copy the full SHA 234d059View commit details -
add README for llamafile integration including setup instruction + no…
…tes on the integration; add helper scripts for downloading/running a llamafile + example env file.
Configuration menu - View commit details
-
Copy full SHA for 05d2b81 - Browse repository at this point
Copy the full SHA 05d2b81View commit details
Commits on Apr 19, 2024
-
simplify mistral message handling; set seed=0 in chat completion kwar…
…gs for reproducibility
Configuration menu - View commit details
-
Copy full SHA for 1cd3e8b - Browse repository at this point
Copy the full SHA 1cd3e8bView commit details -
set mistral max_tokens to actual value configured in the model and ch…
…ange serve.sh to use model's full context size (this does not seem to cause OOM errors, surpisingly).
Configuration menu - View commit details
-
Copy full SHA for dc36c69 - Browse repository at this point
Copy the full SHA dc36c69View commit details -
Configuration menu - View commit details
-
Copy full SHA for e426766 - Browse repository at this point
Copy the full SHA e426766View commit details
Commits on May 25, 2024
-
Configuration menu - View commit details
-
Copy full SHA for d63aa23 - Browse repository at this point
Copy the full SHA d63aa23View commit details -
Configuration menu - View commit details
-
Copy full SHA for 7e7037d - Browse repository at this point
Copy the full SHA 7e7037dView commit details
Commits on May 30, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 3c1f283 - Browse repository at this point
Copy the full SHA 3c1f283View commit details
Commits on May 31, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 5d0f8b0 - Browse repository at this point
Copy the full SHA 5d0f8b0View commit details -
Create
BaseOpenAIProvider
with common functionality from `OpenAIPro……vider`, `GroqProvider` and `LlamafileProvider` and rebase the latter three on `BaseOpenAIProvider`
Configuration menu - View commit details
-
Copy full SHA for 960155a - Browse repository at this point
Copy the full SHA 960155aView commit details
Commits on Jun 2, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 7aed930 - Browse repository at this point
Copy the full SHA 7aed930View commit details
Commits on Jun 3, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 02d0691 - Browse repository at this point
Copy the full SHA 02d0691View commit details -
Configuration menu - View commit details
-
Copy full SHA for f53c2de - Browse repository at this point
Copy the full SHA f53c2deView commit details -
Configuration menu - View commit details
-
Copy full SHA for f78ad94 - Browse repository at this point
Copy the full SHA f78ad94View commit details -
Configuration menu - View commit details
-
Copy full SHA for 1a00ecf - Browse repository at this point
Copy the full SHA 1a00ecfView commit details -
Configuration menu - View commit details
-
Copy full SHA for 3c8bf3c - Browse repository at this point
Copy the full SHA 3c8bf3cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 65433ba - Browse repository at this point
Copy the full SHA 65433baView commit details