[BUG] [AiServices] OllamaStreamingChatModel partial response deserialization throws MalformedJsonException

[kchaber]

Describe the bug
The com.google.gson.stream.MalformedJsonException exception is thrown during TokenStream consumption when using the OllamaStreamingChatModel through the AiServices.

Log and Stack trace

com.google.gson.JsonSyntaxException: com.google.gson.stream.MalformedJsonException: Use JsonReader.setLenient(true) to accept malformed JSON at line 2 column 2 path $
	at com.google.gson.Gson.assertFullConsumption(Gson.java:908)
	at com.google.gson.Gson.fromJson(Gson.java:898)
	at com.google.gson.Gson.fromJson(Gson.java:846)
	at com.google.gson.Gson.fromJson(Gson.java:817)
	at dev.langchain4j.model.ollama.OllamaClient$2.onResponse(OllamaClient.java:132)
	at retrofit2.OkHttpCall$1.onResponse(OkHttpCall.java:161)
	at okhttp3.internal.connection.RealCall$AsyncCall.run(RealCall.kt:519)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: com.google.gson.stream.MalformedJsonException: Use JsonReader.setLenient(true) to accept malformed JSON at line 2 column 2 path $
	at com.google.gson.stream.JsonReader.syntaxError(JsonReader.java:1564)
	at com.google.gson.stream.JsonReader.checkLenient(JsonReader.java:1405)
	at com.google.gson.stream.JsonReader.doPeek(JsonReader.java:543)
	at com.google.gson.stream.JsonReader.peek(JsonReader.java:426)
	at com.google.gson.Gson.assertFullConsumption(Gson.java:904)
	... 9 more

To Reproduce

public interface StreamingAssistant {

    TokenStream chat(String userMessage);

}

OllamaStreamingChatModel ollamaModel = OllamaStreamingChatModel.builder()
    .baseUrl("<url>")
    .modelName("mistral")
    .build();

StreamingAssistant assistant = AiServices.builder(StreamingAssistant.class)
    .streamingChatLanguageModel(ollamaModel)
    .build();

assistant.chat("Tell me a joke")
    .onNext(System.out::println)
    .onComplete(System.out::println)
    .onError(Throwable::printStackTrace)
    .start();

Expected behavior
No exception is thrown when processing streaming response.

Please complete the following information:

• LangChain4j version: 0.30.0
• LLM(s) used: ollama mistral
• Java version: 17
• Spring Boot version (if applicable):

Additional context
The exception is thrown in the OllamaClient when the partial response does not contain valid json as presented in the below debug:

[langchain4j]

Hi, it should be already fixed by this.
Could you please check snapshot version?

<dependency>
  <groupId>dev.langchain4j</groupId>
  <artifactId>langchain4j-ollama</artifactId>
  <version>0.31.0-SNAPSHOT</version>
</dependency>

[kchaber]

Hi @langchain4j, I've checked the snapshot version but it still does not work.
However it seems that mentioned change is not included in the snapshot build. I can see that last build was on 16/04 where #867 was merged on 19th and is not included in the artifact.

[langchain4j]

Hi @kchaber, thanks for the heads up!
I have fixed snapshot build, 0.31.0-SNAPSHOT can be used now.

[langchain4j]

@kchaber please close this issue if it worked for you 🙏

[kchaber]

@langchain4j yes, that works correctly now ☺️ Thank you!