Welcome to langchain4j Discussions!

[langchain4j]

👋 Welcome!

We’re using Discussions as a place to connect with other members of our community. We hope that you:

• Ask questions you’re wondering about
• Suggest improvements to the library
• Share ideas
• Engage with other community members
• Welcome others and are open-minded. Remember that this is a community we
  build together 💪

Please share your ideas below!

[Preetam-Blr]

Hi Just want to know how to get number of tokens in Aiservice

[LizeRaes]

A TokenCountEstimator can normally be found on the model implementation (not yet on the interface in this release). So you create your model to use in AIService, on the model itself (implementation) you find token estimator, see example devoxx / Streaming. Hope it helps!

[Preetam-Blr]

@LizeRaes which is the stable version to be used?

[langchain4j]

You can find the latest version here, currently it is 0.23.0

[langchain4j]

@Preetam-Blr to get the number of tokens used in AiServices, you can define Response<AiMessage> as a return type, like this:

interface Assistant {

        Response<AiMessage> chat(String message);
}

Assistant assistant = AiServices.create(Assistant.class, model);
Response<AiMessage> response = assistant.chat("Hello");
System.out.println(response.tokenUsage());

If you prefer to have streaming, you can define TokenStream as a return type:

interface Assistant {

        TokenStream chat(String message);
}

assistant.chat("Hello")
    .onNext(token -> System.out.println(token))
    .onComplete(response -> System.out.println(response.tokenUsage()))
    .ignoreErrors()
    .start();

[Preetam-Blr]

@langchain4j the blow is the gradle dependency right?
implementation 'dev.langchain4j:langchain4j:0.23.0'

[langchain4j]

yes, this imports common functionality of langchain4j (like prompt templates, memory, etc), but you also need to import specific model providers like OpenAI/VertexAI, they come in their separate modules: langchain4j-open-ai, langchain-vertex-ai, etc. Same goes for embedding store implementations, they all live in their separate modules

[Preetam-Blr]

not able to import AzureOpenAiChatModel

[langchain4j]

use dev.langchain4j:langchain4j-azure-open-ai:0.23.0

[Preetam-Blr]

thanks its working now

[Preetam-Blr]

@langchain4j Can system message be build using structured prompt like user message?

[langchain4j]

@Preetam-Blr yes, StructuredPromptProcessor.toPrompt(structuredPromptObject).toSystemMessage()

[ice09]

I would like to ask some questions about the embeddings, but they are not langchain4j specific. I am trying to find out how many embeddings are possible? Is there a way to get the used context/token size? I am using the default mechanism (cmp. https://gist.github.com/ice09/ae5409de706fbd5e08183f7632b44434)

[Preetam-Blr]

@langchain4j @LizeRaes

interface Assistant {

    Response<AiMessage> chat(List<String> message);

}
Can we send userMessage and Systemmessage as a list of string ?

[langchain4j]

Not with AI services, but you can do so using one of ChatLanguageModel implementations. How many messages are in the list? If only system message and user message, you can do fine with just annotations above this method, see examples

[Preetam-Blr]

@langchain4j I have to do with Structured prompt

[Preetam-Blr]

I have to pass both Systemmessage and Usermessage as Structured prompt

[langchain4j]

Look for Recipe createRecipe(CreateRecipePrompt prompt); example here, you can also annotate this method with @SystemMessage

[Preetam-Blr]

If there is any place holder in @SystemMessage how to populate that value using Structured prompt?

[Preetam-Blr]

@langchain4j I have one doubt.I want the result from the GPT in a consistent JSON or YAML format How can I achieve it using function call that is tool in langchain4j?Please help