[Bug]: guided jsons with date fields are not valid

[andreas-22]

Your current environment

I host llama3-8B-instruct model
in a docker image using the base image vllm/vllm-openai:0.4.1

openai version: 1.6.1 (example script below)

🐛 Describe the bug

description

When we prompt the model to produce a json with the guided_json parameter set and including a date field, we get an invalid json response.

This is also not dependent on the specific model. We got the same error with mixtral models.

code

from openai import OpenAI


model_name = "/docker_share/models/llama3_8b_instruct"
num_prompt_tokens = 1000
base_url = "http://model-url"
api_key = "EMPTY"
client = OpenAI(base_url=base_url, api_key=api_key)
def call_vllm(prompt, guided_json=None):
        if guided_json:
            extra_body = {
                 "guided_json": guided_json
                 }
        messages = [{"role": "user", "content": prompt}]
        
        chat_completion = client.chat.completions.create(
            model=model_name,
            messages=messages,
            extra_body=extra_body,
            seed=42,
            max_tokens=num_prompt_tokens,
            temperature=0
        )
        return chat_completion.choices[0].message.content

guided_json = """{ 
    "additionalProperties": false, 
    "properties": { 
        "DocumentDate": {"anyOf": [{"format": "date", "type": "string"}, {"type": "null"}], 
        "description": "Document date", 
        "title": "DocumentDate"} 
    }, 
    "required": ["DocumentDate"], 
    "title": "Document", 
    "type": "object" 
}"""

if __name__ == "__main__":
    res = call_vllm("Extract a date from this string: date of the document - 01.01.1999", guided_json=guided_json)
    print(res)

result

{ "DocumentDate": 1999-01-01 }

expected result

{ "DocumentDate": "1999-01-01" }

[simon-mo]

can you reproduce this with just outlines https://github.com/outlines-dev/outlines?tab=readme-ov-file#efficient-json-generation-following-a-json-schema

[andreas-22]

This seems to be related to a known issue with older versions of outlines:
bug: predibase/lorax#392
fix: outlines-dev/outlines#567

Perhaps the outlines version should be updated
In vllm, in requirements-common.txt it is specified outlines == 0.0.34.
Using outlines==0.0.41 in the example script above, I get valid jsons with date field.

[simon-mo]

Being worked on in #4558 and #4330