Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat/openai n #5087

Closed
wants to merge 6 commits into from
Closed

Feat/openai n #5087

wants to merge 6 commits into from

Conversation

jacoblee93
Copy link
Collaborator

@davidfant @functorism this is roughly what it would take to have the same featureset as .batch (separate runs per input, with error handling).

I think the maintenance overhead would be tough - would you all feel comfortable calling .generate() directly instead?

@dosubot dosubot bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Apr 13, 2024
Copy link

vercel bot commented Apr 13, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
langchainjs-api-refs ✅ Ready (Inspect) Visit Preview 💬 Add feedback Apr 13, 2024 0:51am
langchainjs-docs ✅ Ready (Inspect) Visit Preview Apr 13, 2024 0:51am

@davidfant
Copy link
Contributor

@jacoblee93 hmm, agree that this doesn't look clean on the ChatOpenAI-side. is there any other good way to accomplish this kind of batching without hacking the batch fn and without using generate? the problem with generate is that I want to use the same interface for across my app when doing LLM calls with an arbitrary BaseChatModel (mostly Claude and GPT-4). i want to avoid if OpenAI, generate in a special way

@jacoblee93
Copy link
Collaborator Author

Yeah I gotcha - unfortunately I think nothing comes to mind CC @baskaryan @eyurtsev :(

I think for now you could wrap it in a custom function? I hear you on wanting a unified interface for sure though.

@jacoblee93 jacoblee93 closed this May 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto:improvement Medium size change to existing code to handle new use-cases size:L This PR changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants