[Question]: Order of the available models are random

[simon-franz]

What is your question?

I am using different models across AWS Bedrock and Azure OpenAI. Hence, I am using LiteLLM as a proxy. Unfortunately, it does not seem possible to specify the order of the listed models. With each restart of the Docker images, the order of the models is shuffled.

The models are specified in a collection as follows:
default: ["gpt-3.5-turbo","gpt-4","claude-v2.1","claude-v3-haiku","claude-v3-sonnet","mistral-7B","mistral-8x7B"]

However, the result seems to be random (see screenshot at the end)

Is there any way to specify the order of the models?

More Details

Given that I have to use LiteLLM because of bedrock and azure and simple expense-tracking I would also like to split up the custom endpoint into 3 different endpoints, Like so

custom:
    - name: "Openai"
      apiKey: "xx"
      baseURL: "http://host.docker.internal:4000"
      models:
        default: ["gpt-3.5-turbo","gpt-4"]
        fetch: true
      modelDisplayLabel: "Openai"
custom:
    - name: "Antrophic"
      apiKey: "xx"
      baseURL: "http://host.docker.internal:4000"
      models:
        default: ["claude-v2.1","claude-v3-haiku","claude-v3-sonnet"]
        fetch: true
      modelDisplayLabel: "Antrophic"
custom:
    - name: "Mistral"
      apiKey: "xx"
      baseURL: "http://host.docker.internal:4000"
      models:
        default: ["mistral-7B","mistral-8x7B"]
        fetch: true
      modelDisplayLabel: "Mistral"

This would also enable to set the correct iconURL:
Maybe I have setup wrong and this is alredy possible?

Thanks in advance for any answer!

What is the main subject of your question?

UI

Screenshots

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

It is possible. The app should honor the order you set in your config. What is your fetch value? because you might be fetching the list from LiteLLM and it might be shuffling the order.

        default: ["gpt-3.5-turbo","gpt-4","claude-v2.1","claude-v3-haiku","claude-v3-sonnet","mistral-7B","mistral-8x7B"]
        fetch: false

I always get the order I set above.

[danny-avila]

you might be making your config file "invalid" when you edit it, could be the way it's formatted. do a quick https://yamlchecker.com/ check because that could really be what's hiding it.

Also make sure you are updating like this:

https://docs.librechat.ai/install/installation/docker_compose_install.html#updating-librechat

[danny-avila]

also check the logs, specifically on LibreChat startup

[simon-franz]

Thanks for your response

Having set fetch to false the startup does display this:

LibreChat-API      | 2024-04-25 19:47:53 info: Custom config file loaded:
LibreChat-API      | 2024-04-25 19:47:53 info: {
LibreChat-API      |   "version": "1.0.5",
LibreChat-API      |   "cache": true,
LibreChat-API      |   "fileConfig": {
LibreChat-API      |     "serverFileSizeLimit": 100,
LibreChat-API      |     "avatarSizeLimit": 4,
LibreChat-API      |     "endpoints": {
LibreChat-API      |       "azureOpenAI": {
LibreChat-API      |         "disabled": true
LibreChat-API      |       }
LibreChat-API      |     }
LibreChat-API      |   },
LibreChat-API      |   "registration": {
LibreChat-API      |     "socialLogins": [
LibreChat-API      |       "google"
LibreChat-API      |     ],
LibreChat-API      |     "allowedDomains": [
LibreChat-API      |       "mydomain.de"
LibreChat-API      |     ]
LibreChat-API      |   },
LibreChat-API      |   "endpoints": {
LibreChat-API      |     "custom": [
LibreChat-API      |       {
LibreChat-API      |         "name": "LiteLLM",
LibreChat-API      |         "apiKey": "sk_live_xxxxxxxxxxxxxxxxxxxxx",
LibreChat-API      |         "baseURL": "http://host.docker.internal:4000",
LibreChat-API      |         "models": {
LibreChat-API      |           "default": [
LibreChat-API      |             "gpt-3.5-turbo",
LibreChat-API      |             "gpt-4",
LibreChat-API      |             "claude-v2.1",
LibreChat-API      |             "claude-v3-haiku",
LibreChat-API      |             "claude-v3-sonnet",
LibreChat-API      |             "mistral-7B",
LibreChat-API      |             "mistral-8x7B"
LibreChat-API      |           ],
LibreChat-API      |           "fetch": false
LibreChat-API      |         },
LibreChat-API      |         "titleConvo": true,
LibreChat-API      |         "titleModel": "gpt-3.5-turbo",
LibreChat-API      |         "summarize": false,
LibreChat-API      |         "summaryModel": "gpt-3.5-turbo",
LibreChat-API      |         "forcePrompt": false,
LibreChat-API      |         "modelDisplayLabel": "myName"
LibreChat-API      |       }
LibreChat-API      |     ]
LibreChat-API      |   }
LibreChat-API      | }

The yaml file doesn't have any erorrs, but the UI doesn't show the model-selection:

[danny-avila]

what is the value of ENDPOINTS in your .env file?

You should either un-comment it or add custom to the list

ENDPOINTS=custom,openAI,azureOpenAI # more endpoints by comma

used and tested successfully:

version: 1.0.7
cache: true
endpoints:
  custom:
    - name: "LiteLLM"
      apiKey: "sk_live_xxxxxxxxxxxxxxx"
      baseURL: "http://host.docker.internal:4000"
      models:
        default: ["gpt-3.5-turbo","gpt-4","claude-v2.1","claude-v3-haiku","claude-v3-sonnet","mistral-7B","mistral-8x7B"]
        fetch: false
      titleConvo: true
      titleModel: "gpt-3.5-turbo"
      summarize: false
      summaryModel: "gpt-3.5-turbo"
      forcePrompt: false
      modelDisplayLabel: "myName"

[simon-franz]

Thank you very much, this was indeed the cause. Commenting out the endpoints fixed the issue.
Now the models are in the given order!