How to create a service (AIServices) with memory and streaming responses?

[rgrin08]

I would like to use a service with memory and streaming.
Code without streaming:

Assistant assistant = AiServices.builder(Assistant.class)
     .chatLanguageModel(model)
     .chatMemory(chatMemory)
     .build();

This code does not work if the type of the model is StreamingChatLanguageModel.
How can I do in this case?

[tenpigs267]

You should just use .streamingChatLanguageModel instead of .chatLanguageModel
or use the constructor as in this example : https://github.com/langchain4j/langchain4j-examples/blob/main/other-examples/src/main/java/ServiceWithStreamingExample.java

[anunnakian]

Maybe this PR will help you #971
I merged the ChatLanguageModel and StreamingChatLanguageModel classes, so you will be able to do this :

OpenAiChatModel.builder()
      .baseUrl(System.getenv("OPENAI_BASE_URL"))
      .apiKey(System.getenv("OPENAI_API_KEY"))
      .organizationId(System.getenv("OPENAI_ORGANIZATION_ID"))
      .logRequests(true)
      .logResponses(true)
      .isStreaming(true) // Enable streaming 
      .build()

[langchain4j]

Hi @rgrin08, you need to define the return type of your AI Service method as TokenStream, please see an example here. You can add memory to it too.