Freature #1005 - Add streaming API for Bedrock Anthropics #1006
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@langchain4j Previous comments were addressed. Would be amazing if you could run CI I checked locally everything compiles and tests pass. |
|
PR doesn't address token count as Bedrock doesn't expose such information as part of response. This will require token estimations which will be addressed in the next PR. |
|
@langchain4j would be amazing if you could give a look on it. |
|
@michalkozminski I will review it asap, in the order of priorities. Thanks for your patience! |
langchain4j
approved these changes
May 7, 2024
| @SuperBuilder | ||
| public class BedrockAnthropicStreamingChatModel extends AbstractBedrockStreamingChatModel { | ||
| @Builder.Default | ||
| private final String model = BedrockAnthropicStreamingChatModel.Types.AnthropicClaudeV2.getValue(); |
There was a problem hiding this comment.
minor: should it use one of v3 models (e.g. haiku) by default?
| @@ -179,7 +179,7 @@ See example [here](https://github.com/langchain4j/langchain4j-examples/blob/main | |||
| | [OpenAI](https://docs.langchain4j.dev/integrations/language-models/open-ai) | ✅ | ✅ | ✅ | ✅ | ✅ | | ✅ | | |||
| | [Azure OpenAI](https://docs.langchain4j.dev/integrations/language-models/azure-open-ai) | | ✅ | ✅ | ✅ | ✅ | | ✅ | | |||
| | [Hugging Face](https://docs.langchain4j.dev/integrations/language-models/hugging-face) | | ✅ | | ✅ | | | | | | |||
| | [Amazon Bedrock](https://docs.langchain4j.dev/integrations/language-models/amazon-bedrock) | | ✅ | | ✅ | ✅ | | | | |||
| | [Amazon Bedrock](https://docs.langchain4j.dev/integrations/language-models/amazon-bedrock) | | ✅ | ✅ | ✅ | ✅ | | | | |||
There was a problem hiding this comment.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Context
Feature adds Bedrock Antrhopics Streaming capability #1005
Previous PR #679
Change
Added new streaming model
AbstractBedrockStreamingChatModelChecklist
Before submitting this PR, please check the following points:
Checklist for adding new embedding store integration