.Net: Function calling abstraction

[SergeyMenshykh]

Motivation and Context

Today, every AI connector in SK that supports function calling has its own implementation of tool call behavior model classes. These classes are used to configure the way connectors advertise and invoke functions. For example, the behavior classes can describe which functions should be advertised to LLM by a connector, whether the functions should be called automatically by the connector, or if the connector caller will invoke them himself manually, etc.

All the tool call behavior classes are the same in terms of describing the desired function call behavior. However, the classes have a mapping functionality to map the function call behavior to the connector-specific model classes, and that's what makes the function calling classes non-reusable between connectors.

Additionally, today, it's not possible to specify function calling behavior declaratively in YAML, MD, and Kernel(config.json) prompts.

Description

This PR introduces a few function choice behavior classes that allow configure function calling behavior in connector agnostic way:

• FunctionChoiceBehavior - an abstract class that used as a base class for all the choice behaviors described below.
• AutoFunctionChoiceBehavior - inherits the FunctionChoiceBehavior class and contains configuration to tell LLM to decide itself whether to call kernel or provided function(s) or not.
• RequiredFunctionChoiceBehavior - inherits the FunctionChoiceBehavior class and contains configuration forcing LLM to call one or more kernel or provided functions.
• NoneFunctionChoiceBehavior - inherits the FunctionChoiceBehavior class and contains configuration to tell the model not to call functions and only generate a user-facing message.

Callers can specify one of the function choice behaviors using the new property, FunctionChoiceBehavior, which has been added to the PromptExecutionSettings class. Here's an example:

PromptExecutionSettings settings = new PromptExecutionSettings() 
{ 
   FunctionChoiceBehavior = FunctionChoiceBehavior.AutoFunctionChoice()
};

var result = await completionService.GetChatMessageContentAsync(chatHistory, settings, kernel);

The function choice behavior can be configured in JSON and YAML prompts as well:

"execution_settings": {
    "service-gpt4": {
       "model_id": "gpt4",
       "temperature": 0.7,
       "function_choice_behavior": {
           "type": "auto",
           "functions": [
              "p1.f1"
           ]
        }
    }
}

execution_settings:
  service-gpt4:
    model_id: gpt-4
    temperature: 1.0
    function_choice_behavior:
      type: auto
      functions:
      - p1.f1

Closes - #5253

ADR - #6099

Contribution Checklist

• The code builds clean without any errors or warnings
• The PR follows the SK Contribution Guidelines and the pre-submission formatting script raises no violations
• All unit tests pass, and I have added new tests where possible
• I didn't break anyone 😄

[dmytrostruk]

Suggested change

	throw new ArgumentException("ToolBehaviors and ToolCallBehavior cannot be used together.");
	throw new ArgumentException($"{nameof(executionSettings.FunctionChoiceBehavior)} and {executionSettings.ToolCallBehavior} cannot be used together.");

I also think that we should not throw an exception and use configuration based on priority. If both are specified, use FunctionChoiceBehavior as a new approach, which will overwrite ToolCallBehavior, we can mention this behavior in XML documentation.

New approach also allows to specify configuration in json/yaml. So, if I have already existing working code with ToolCallBehavior, and I want to try new configuration in json/yaml, I will receive an exception at first try, because it will force me to remove ToolCallBehavior from my existing code. Choosing one property based on priority will require minimal changes from developers.

[SergeyMenshykh]

I also think that we should not throw an exception and use configuration based on priority. If both are specified, use FunctionChoiceBehavior as a new approach, which will overwrite ToolCallBehavior, we can mention this behavior in XML documentation.

Both solutions look good to me. The one that throws the exception notifies the caller immediately when the code is invoked, potentially saving time they would spend troubleshooting unexpected behavior. The one that relies on priority is more permissive and allows callers to use both solutions simultaneously, even at the cost of potential confusion or unexpected behavior. I would probably go with the proposed "priority" one taking into account its permissiveness. Let's see what others think.

New approach also allows to specify configuration in json/yaml. So, if I have already existing working code with ToolCallBehavior, and I want to try new configuration in json/yaml, I will receive an exception at first try, because it will force me to remove ToolCallBehavior from my existing code. Choosing one property based on priority will require minimal changes from developers.

Today, as far as I understand, prompt execution settings provided via the code override the ones that come from prompts, so no exception will be thrown in this scenario. To allow execution settings specified in the prompt to be applied, either the settings specified in the code should be removed or a different service selector should be used.

[dmytrostruk]

Is it a limitation in OpenAI only or the same is applicable in other connectors? It would be nice to see this limitation during compilation, rather than runtime, if possible of course.

[SergeyMenshykh]

Is it a limitation in OpenAI only or the same is applicable in other connectors?

• OpenAI initially supported only one required function, but recently added support for multiple functions - https://community.openai.com/t/new-api-feature-forcing-function-calling-via-tool-choice-required/731488. However, we cannot use it yet because the Azure.AI.OpenAI SDK has not exposed the new tool choice yet.
• Mistral AI seems to support only one - https://docs.mistral.ai/capabilities/function_calling/#tool_choice
• Gemini supports many - https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/function-calling#tool-config

It would be nice to see this limitation during compilation, rather than runtime, if possible of course.

It's an interesting problem to solve. Without going into the design, I see two options:

• To have a set of choices that can be specified on a connector-specific prompt execution setting, so that the choices' usage can be enforced by the compiler.
• To have a set of choices that can be supplied via the connector constructor and enforced by compiler as well.
  Both have their pros and cons and would require another iteration of design.

We can also consider other solutions to address the problem:

• Use the first function from the list - this is not obvious and can be confusing for callers.
• Execute the functions by the connector itself - this will solve the problem but will require a solution for obtaining arguments for the functions.

[dmytrostruk]

Logic in this method is very similar to logic in AutoFunctionChoiceBehavior.GetConfiguration method. Should we move it to the base class and re-use?