Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(forge/llm): Add LlamafileProvider #7091

Open
wants to merge 20 commits into
base: master
Choose a base branch
from

Commits on Apr 18, 2024

  1. Add minimal implementation of LlamafileProvider, a new ChatModelProvi…

    …der for llamafiles. Currently it just extends OpenAIProvider and only overrides methods that are necessary to get the system to work at a basic level.
    
    Update ModelProviderName schema and config/configurator so that app startup using this provider is handled correctly.
    Add 'mistral-7b-instruct-v0' to OpenAIModelName/OPEN_AI_CHAT_MODELS registries.
    k8si committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    03d8e1e View commit details
    Browse the repository at this point in the history
  2. Adapt model prompt message roles to be compatible with the Mistral-7b…

    …-Instruct chat template, which supports the 'user' & 'assistant' roles but does not support the 'system' role.
    k8si committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    ed1dfd0 View commit details
    Browse the repository at this point in the history
  3. In OpenAIProvider, change methods count_message_tokens, `count_to…

    …kens`, and `get_tokenizer` from classmethods so I can override them in LlamafileProvide (and so I can access instance instance attributes from inside them). Implement class `LlamafileTokenizer` that calls the llamafile server's `/tokenize` API endpoint.
    k8si committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    c56c290 View commit details
    Browse the repository at this point in the history
  4. misc cleanup

    k8si committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    234d059 View commit details
    Browse the repository at this point in the history
  5. add README for llamafile integration including setup instruction + no…

    …tes on the integration; add helper scripts for downloading/running a llamafile + example env file.
    k8si committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    05d2b81 View commit details
    Browse the repository at this point in the history

Commits on Apr 19, 2024

  1. simplify mistral message handling; set seed=0 in chat completion kwar…

    …gs for reproducibility
    k8si committed Apr 19, 2024
    Configuration menu
    Copy the full SHA
    1cd3e8b View commit details
    Browse the repository at this point in the history
  2. set mistral max_tokens to actual value configured in the model and ch…

    …ange serve.sh to use model's full context size (this does not seem to cause OOM errors, surpisingly).
    k8si committed Apr 19, 2024
    Configuration menu
    Copy the full SHA
    dc36c69 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    e426766 View commit details
    Browse the repository at this point in the history

Commits on May 25, 2024

  1. Configuration menu
    Copy the full SHA
    d63aa23 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    7e7037d View commit details
    Browse the repository at this point in the history

Commits on May 30, 2024

  1. Configuration menu
    Copy the full SHA
    3c1f283 View commit details
    Browse the repository at this point in the history

Commits on May 31, 2024

  1. fix linting errors

    Pwuts committed May 31, 2024
    Configuration menu
    Copy the full SHA
    5d0f8b0 View commit details
    Browse the repository at this point in the history
  2. Create BaseOpenAIProvider with common functionality from `OpenAIPro…

    …vider`, `GroqProvider` and `LlamafileProvider`
    
    and rebase the latter three on `BaseOpenAIProvider`
    Pwuts committed May 31, 2024
    Configuration menu
    Copy the full SHA
    960155a View commit details
    Browse the repository at this point in the history

Commits on Jun 2, 2024

  1. Configuration menu
    Copy the full SHA
    7aed930 View commit details
    Browse the repository at this point in the history

Commits on Jun 3, 2024

  1. Configuration menu
    Copy the full SHA
    02d0691 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    f53c2de View commit details
    Browse the repository at this point in the history
  3. clean up llamafile readme

    Pwuts committed Jun 3, 2024
    Configuration menu
    Copy the full SHA
    f78ad94 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    1a00ecf View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    3c8bf3c View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    65433ba View commit details
    Browse the repository at this point in the history