.Net: The operation was cancelled because it exceeded the configured timeout of 0:01:40

[takeo-iw]

I'm using Azure OpenAI with native plugin, but I got an error "The operation was cancelled because it exceeded the configured timeout of 0:01:40. Network timeout can be adjusted in ClientOptions.Retry.NetworkTimeout.".

I fixed it using the same method referring to the issue below, and the error was resolved.
#3223

Before the fix, I was utilizing RetryHandler but it didn't work.

So, I have a question.
Does RetryHandler not apply to Azure OpenAI connectors?
In that case, if the Connector(LLM) changes, the coding will have to change again, it's increasing dependence.

[evchaki]

@takeo-iw - it should apply to this. Adding @RogerBarreto to add in any additional info. Also check out 1.0 (https://devblogs.microsoft.com/semantic-kernel/semantic-kernel-v1-0-1-has-arrived-to-help-you-build-agents/) to see if that solves your problem.

[matthewbolanos]

To align with services that are used elsewhere in the .NET ecosystem, Semantic Kernel has adopted the main packages/patterns used by apps that use Dependency Injection. To get ConfigureHttpClientDefaults, you'll want to make sure you include the following:

using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Http.Resilience;

Because of how critical this scenario is, we'll be updating our docs to make it more clear how to set retry logic with Microsoft's Dependency Injection .

[Alerinos]

Thanks for the help, I did not have the Microsoft.Extensions.Http.Resilience package installed and Visual Studio did not prompt but threw an error.

This is how I did the timeout workaround by the way I was getting error 502 sometimes

        builder.Services.ConfigureHttpClientDefaults(c =>
        {
            // Use a standard resiliency policy, augmented to retry on 401 Unauthorized, 502 Bad Gateway and on timeout for this example
            c.AddStandardResilienceHandler().Configure(o =>
            {
                // Combine checks for status codes and timeout into one delegate to set as the ShouldHandle policy
                o.Retry.ShouldHandle = args =>
                {
                    // Check if the response has a status code that we should handle
                    var isStatusCodeToRetry = args.Outcome.Result?.StatusCode is HttpStatusCode.Unauthorized
                                              || args.Outcome.Result?.StatusCode is HttpStatusCode.BadGateway;

                    // Check if the result was a timeout (typically when the result is null and an exception is a TaskCanceledException)
                    var isTimeout = args.Outcome.Exception?.InnerException is TaskCanceledException;

                    // Combine the checks
                    return ValueTask.FromResult(isStatusCodeToRetry || isTimeout);
                };
            });
        });

[takeo-iw]

I got the bellow error.

Unhandled exception. Polly.Timeout.TimeoutRejectedException: The operation didn't complete within the allowed timeout of '00:00:10'.
 ---> System.Threading.Tasks.TaskCanceledException: The operation was canceled.

How do I set Timeout policy?

[Alerinos]

@takeo-iw
I did such a thing, although I have the impression that the 10-minute time is ignored and sometimes I get 1 minute and 20 or 40 seconds.

    public Kernel CreateChatContext(string token, string model)
    {
        var httpClient = new HttpClient
        {
            Timeout = TimeSpan.FromMinutes(10),
        };

        var builder = Kernel.CreateBuilder();

        builder.Services.ConfigureHttpClientDefaults(c =>
        {
            // Use a standard resiliency policy, augmented to retry on 401 Unauthorized, 502 Bad Gateway and on timeout for this example
            c.AddStandardResilienceHandler().Configure(o =>
            {
                // Combine checks for status codes and timeout into one delegate to set as the ShouldHandle policy
                o.Retry.ShouldHandle = args =>
                {
                    // Check if the response has a status code that we should handle
                    var isStatusCodeToRetry = args.Outcome.Result?.StatusCode is HttpStatusCode.Unauthorized
                                              || args.Outcome.Result?.StatusCode is HttpStatusCode.BadGateway;

                    // Check if the result was a timeout (typically when the result is null and an exception is a TaskCanceledException)
                    var isTimeout = args.Outcome.Exception?.InnerException is TaskCanceledException;

                    // Combine the checks
                    return ValueTask.FromResult(isStatusCodeToRetry || isTimeout);
                };
            });
        });

        builder.AddOpenAIChatCompletion(
            model,
            token,
            httpClient: httpClient
        );

        return builder.Build();
    }

[takeo-iw]

@Alerinos

Thank you. solved.