New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Retry strategy on rate limit #1490
Comments
@dblock Thanks for reaching out about this! ❤ Regarding rate limits, currently, the SDK doesn't have a great way to bubble that up to you. While the rate limit information is present in the HTTP headers of the request coming back from the REST API call, no logic or method calculates the "cool down time." Take a look at client/rate_limit.rb and rate_limit.rb to see if any implementations in there might help. Given that we don't yet support generative models and do not have pagination implemented, we'd welcome any contributions to paging that you'd be providing. Please feel free to submit a PR with your thoughts about auto-pagination. |
Coming from #1489 I am attempting to handle the rate limit as follows.
There are a few problems here.
rate_limit.resets_in
value does not seem like the right amount of time to wait. It is the entire duration till the next reset, even before I have consumed all requests. How do I obtain the time when I will be allowed to make the next request, aka the cool down time?The text was updated successfully, but these errors were encountered: