Create a Template class to standardize prompt styles

[shengyangs]

Is your feature request related to a problem? Please describe.

Different models rely on difference prompt styles (system prompt, extra-id, byte-token, steemlm scores). Each model needs to match its prompt styles exactly to avoid performance degradation, which is done by manual string concatenations. The current approach is error-prone.

Describe the solution you'd like

We want to implement a class for Prompt Templates. For each prompt style, orchestrating its (prompt, response) needs only calling the formatting function in the template. This is akin to the HuggingFace design.

An example design of the class is as follows,

class PromptTemplate:
    def __init__(self):
        self.name = "a_very_simple_prompt"

    @property
    def system_prompt(self):
        return "System: \n\n"

    def _generation_msg(self, role):
        return f"{role}:"

    def format_single_msg(self, role, msg):
        if role == "User":
            return "User: {msg}\n\n"
        elif role == 'Assistant':
            return "Assistant: {msg}\n\n"
        else:
            raise NotImplementedError

    def format_messages(self, messages, append_generation_role=None):
         results = self.system_prompt
         for role, msg in messages.items():
             results = results + self.format_single_msg(role, msg)
         if append_generation_role:
             results = results + self._generation_msg(role=append_generation_role)
         return results

Describe alternatives you've considered

A clear and concise description of any alternative solutions or features you've considered.

Additional context

Add any other context or screenshots about the feature request here

[odelalleau]

There are links with another initiative. Will reach out offline.