The chatml template doesn't have a system message in the tokenizer_config.json

[snehith-r]

System Info

I don't see any system message in the chatml template of tokenizer_confi.json but the example in the docs gives system message as well in the messages,is this a mistake does phi3-instruct doesn't take a system message? even when I tried the same example give here https://huggingface.co/docs/transformers/main/model_doc/phi3 I don't see any system tags in the output.

"chat_template": "{{ bos_token }}{% for message in messages %}{% if (message['role'] == 'user') %}{{'<|user|>' + '\n' + message['content'] + '<|end|>' + '\n' + '<|assistant|>' + '\n'}}{% elif message['role'] == 'assistant' %}{{message['content'] + '<|end|>' + '\n'}}{% endif %}{% endfor %}"

Who can help?

@ArthurZucker @youne

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("microsoft/Phi-3-mini-4k-instruct")
tokenizer = AutoTokenizer.from_pretrained("microsoft/Phi-3-mini-4k-instruct")

messages = [{"role": "system", "content": "You are a helpful digital assistant. Please provide safe, ethical and accurate information to the user."},{"role": "user", "content": "Can you provide ways to eat combinations of bananas and dragonfruits?"}]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")

outputs = model.generate(inputs, max_new_tokens=32)
text = tokenizer.batch_decode(outputs)[0]
print(text)
Output:
<|user|>
Can you provide ways to eat combinations of bananas and dragonfruits?<|end|>
<|assistant|>
Certainly! Bananas and dragonfruits can be combined in various delicious ways. Here are some ideas for eating combinations of bananas and

Expected behavior

<|system|> You are a helpful digital assistant. Please provide safe, ethical and accurate information to the user.<|end|><|user|> Can you provide ways to eat combinations of bananas and dragonfruits?<|end|><|assistant|> Absolutely! Bananas and dragonfruits are both delicious fruits that can be combined in various ways to create tasty and nutrit

[amyeroberts]

cc @Rocketknight1

[Rocketknight1]

This looks valid - the model's chat template doesn't support system messages, but the examples show system messages, and the model does have a <|system|> token. Will try to grab someone who worked on the port!

[Rocketknight1]

Hi @snehith-r, we confirmed with Microsoft that Phi-3 was not trained with system messages and therefore doesn't understand them. The chat template is correct, and the examples were wrong. They should now be fixed - thanks for reporting it!