Enable streaming option in the OpenAI API server

[adk9]

Now that token streaming support has merged (#397), we can enable streaming response in the OpenAI RESTful API endpoint.

This PR

• adds missing package dependencies for OpenAI API server (fixes Is openai compatible server still working? #459)
• re-enables streaming responses for the OpenAPI API endpoint

Running the Server

python -m mii.entrypoints.openai_api_server \
    --model "mistralai/Mistral-7B-Instruct-v0.1" \
    --port 3000 \
    --host 0.0.0.0

Client

from openai import OpenAI

client = OpenAI(
    base_url="http://ip:port/v1",
    api_key="test",
)

completion = client.chat.completions.create(
    model="mistralai/Mistral-7B-v0.1",
    messages=[
        {
            "role": "user",
            "content": "Tell me a joke.",
        },
    ],
    max_tokens=1024,
    stream=True
)

for chunk in completion:
    if chunk.choices[0].delta.content is not None:
        print(chunk.choices[0].delta.content, end="")

[loadams]

FYI @adk9 - we're trying to get the Pydantic update merged, hopefully very soon. So I was curious on what issues you ran into that you needed to pin this?

[adk9]

FYI @adk9 - we're trying to get the Pydantic update merged, hopefully very soon.

Oh great, thanks for the pointer! I was just starting to look into this myself.

So I was curious on what issues you ran into that you needed to pin this?

I see a 400 Bad Request error.

Error code: 400 - {'object': 'error', 'message': "[{'type': 'missing', 'loc': ('query', 'request'), 'msg': 'Field required', 'input': None}]", 'code': 40001}

I think the issue is with fastapi not fully supporting working with pydantic.v1 while having pydantic v2 installed (tiangolo/fastapi#10360). So the workaround is to either pin fastapi or pydantic to an older version, and I chose to do the latter.