[Bug]: Azure OpenAI assistants need to be initialised before you can send them files

[maxesse]

What happened?

If I start a chat with an Azure OpenAI assistant and attach a file in the first message I get an error back, whereas if I send a Hello beforehand, it works.

Steps to Reproduce

1. Create an assistant with code interpreter, the model doesn't matter
2. Send a file immediately and see the error
3. Send a 'hello' beforehand and the assistant runs correctly

What browsers are you seeing the problem on?

Firefox, Chrome, Safari, Microsoft Edge

Relevant log output

2024-04-24T12:03:35.168Z debug: [/assistants/chat/] req.body
{
  text: "What's in this file?",
  sender: "User",
  isCreatedByUser: true,
  parentMessageId: "00000000-0000-0000-0000-000000000000",
  messageId: "c2778f9a-044b-4644-8eb1-3d698e08502d",
  error: false,
  // 1 file(s)
  files: [{"file_id":"assistant-eb6XkH8DdRWSs21SH3fiq2B5","filepath":"https://instance.openai.azure.com/open... [truncated]],
  model: "gpt-3.5-turbo-1106",
  assistant_id: "asst_qJyX4Z4WC68EqvPrLUtomUjK",
  endpoint: "assistants",
  isContinued: false,
    endpointOption.endpoint: "assistants",
    endpointOption.promptPrefix: undefined,
    endpointOption.assistant_id: "asst_qJyX4Z4WC68EqvPrLUtomUjK",
    modelOptions.model: "gpt-3.5-turbo-1106",
    // 17 openAI(s)
    modelsConfig.openAI: ["gpt-3.5-turbo-0125","gpt-3.5-turbo-16k-0613","gpt-3.5-turbo-16k","gpt-4-turbo-preview","gpt-4-0125-preview","gpt-4-1106-preview","gpt-3.5-turbo","gpt-3.5-turbo-1106","gpt-4-vision-preview","gpt-4","gpt-3.5-turbo-instruct-0914","gpt-3.5-turbo-0613","gpt-3.5-turbo-0301","gpt-3.5-turbo-instruct","gpt-4-0613","text-davinci-003","gpt-4-0314"],
    // 12 google(s)
    modelsConfig.google: ["gemini-pro","gemini-pro-vision","chat-bison","chat-bison-32k","codechat-bison","codechat-bison-32k","text-bison","text-bison-32k","text-unicorn","code-gecko","code-bison","code-bison-32k"],
    // 10 anthropic(s)
    modelsConfig.anthropic: ["claude-3-opus-20240229","claude-3-sonnet-20240229","claude-3-haiku-20240307","claude-2.1","claude-2","claude-1.2","claude-1","claude-1-100k","claude-instant-1","claude-instant-1-100k"],
    // 4 gptPlugin(s)
    modelsConfig.gptPlugins: ["gpt-4-1106-preview","gpt-4-vision-preview","gpt-3.5-turbo-1106","whisper"],
    // 4 azureOpenAI(s)
    modelsConfig.azureOpenAI: ["gpt-4-1106-preview","gpt-4-vision-preview","gpt-3.5-turbo-1106","whisper"],
    // 2 bingAI(s)
    modelsConfig.bingAI: ["BingAI","Sydney"],
    // 2 chatGPTBrowser(s)
    modelsConfig.chatGPTBrowser: ["text-davinci-002-render-sha","gpt-4"],
    // 4 assistant(s)
    modelsConfig.assistants: ["gpt-4-1106-preview","gpt-4-vision-preview","gpt-3.5-turbo-1106","whisper"],
    endpointOption.attachments: [object Promise],
}
2024-04-24T12:03:37.760Z error: [/assistants/chat/] 500 The server had an error processing your request. Sorry about that! You can retry your request, or contact us through our help ... [truncated]
2024-04-24T12:03:37.763Z debug: [/assistants/chat/] Request aborted on close

Screenshots

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

I think it's an Azure side error, based on the error log:

2024-04-24 08:13:35 error: [/assistants/chat/] 500 The server had an error processing your request. Sorry about that! You can retry your request, or contact us through our help center at [email protected] if you keep seeing this error. 

A user reported issues on the Azure playground as well:

Even though I can get past the first step in the thread initialization, it's failing for me in azure assistants playground:

[maxesse]

That user was also me, this morning haha. I think the two things are independent though. I had the issue with 'initialising' the assistant throughout this week, whereas the ability for assistants to process attachments is a new one from yesterday afternoon. They also seem to fail if you try to enable vision or tools at the moment. Maybe let's wait until they restore functionality, to isolate the two things. At least it's not just me with that problem though, seems to be broken for you as well.

[danny-avila]

Oh gotcha, yeah I mean nothing has changed in the handling. This worked before and now does not, and should have parity with the OpenAI, non-azure side:

**edit: wrong screenshot

[maxesse]

Is that screenshot from now, or from when things worked?

[danny-avila]

it's from now, but not using Azure