-
-
Notifications
You must be signed in to change notification settings - Fork 469
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[OpenAI] Instructor for Batch Requests #604
Labels
enhancement
New feature or request
Comments
phatlast96
changed the title
[OpenAI] Instructor for Bulk Requests
[OpenAI] Instructor for Batch Requests
Apr 16, 2024
This feature should make instructor even more powerful. Having access to OpenAI batch requests allows a 50% discount. The API is something like this from openai import OpenAI
client = OpenAI()
client.batches.create(
input_file_id="file-abc123",
endpoint="/v1/chat/completions",
completion_window="24h"
) Clearly there is some difference to be supported, but we'd all love the batch feature to be implemented by instructor |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Is your feature request related to a problem? Please describe.
Since OpenAI has a bulk feature now and I've been using Instructor's JSON formatting feature a lot, I'd love to combine those 2 since I've been hitting my rate rate limit on the number of requests.
Describe the solution you'd like
I would love use Instructor for JSON formatting of bulk requests. This would be beneficial since it's a lot of cost savings.
Describe alternatives you've considered
Continue hitting my rate limit using Instructor.
Additional context
Add any other context or screenshots about the feature request here.
https://community.openai.com/t/batchapi-is-now-available/718416
The text was updated successfully, but these errors were encountered: