How to use langfuse in langchain-chatchat

[daocaorenzzl]

LANGFUSE_SECRET_KEY="my_langfuse_secret_key"
LANGFUSE_PUBLIC_KEY="my_langfuse_public_key"
LANGFUSE_HOST="my_langfuse_host"

By setting the variables above, langfuse can be used normally in langserve or langchain tasks, but not in langchain-chatchat.

Given that langchain-chatchat can be integrated with langsmith, I tried to set the langfuse address in the following variable, but it still doesn't work.
LANGCHAIN_ENDPOINT=“my_langfuse_addr”
LANGCHAIN_API_KEY="my_langfuse_public_key"
LANGCHAIN_PROJECT="langfuse_project_name"

In the langchain task and langserve task, to enable the effect of langfuse, it is necessary to use CallbackHandler(). By reviewing the source code, it was found that AsyncIteratorCallbackHandler and ConversationCallbackHandler are used in langchain-chatchat. However, there is no switch in the source code that allows langchain-chatchat to create CallbackHandler().

callbacks:[<langchain.callbacks.streaming_aiter.AsyncIteratorCallbackHandler object at 0x7feca8215150>, <server.callback_handler.conversation_callback_handler.ConversationCallbackHandler object at 0x7feca8e70c10>]

Can someone guide me on how to use langfuse in langchain-chatchat?

[dosubot[bot]]

Based on the information you've provided, it seems like you've set the environment variables correctly. However, it's important to ensure that these variables are being correctly accessed within the Langchain-Chatchat application.

In Python, environment variables can be accessed using the os module. For example, to access the LANGFUSE_SECRET_KEY variable, you would use os.environ['LANGFUSE_SECRET_KEY']. If the variable is not set, this will raise a KeyError. To avoid this, you can use os.environ.get('LANGFUSE_SECRET_KEY'), which will return None if the variable is not set.

If you're sure that the environment variables are being set and accessed correctly, the issue might be with how Langchain-Chatchat is using these variables to connect to Langfuse. Without more information about the specific error you're encountering, it's difficult to provide a more precise solution.

If you're seeing an error message, could you please provide the exact text of the message? This could help in diagnosing the issue. Additionally, could you provide more information about how you're running Langchain-Chatchat? For example, are you running it in a Docker container, or directly on your machine? This information could also be helpful in identifying the problem.

As for the server/llm_api_stale.py file, it's not clear how this file is related to your issue. This file appears to be a standalone script for launching a service, and it doesn't seem to interact with Langfuse or use the environment variables you've mentioned. If this file is relevant to your issue, could you please provide more information about how you're using it?

Finally, while I strive to provide accurate and helpful information, there may be a better answer that I wasn't able to find. If you're still having trouble after trying the suggestions above, please provide more details about your issue and I'll do my best to help you resolve it.

Sources

• server/llm_api_stale.py

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[daocaorenzzl]

Thank you for your answer. The above three variables did take effect in the operating system. The problem has been solved by modifying the source code of langchain-chatchat.

Make sure that LANGFUSE_SECRET_KEY, LANGFUSE_PUBLIC_KEY, and LANGFUSE_HOST are correctly configured in the system environment, and then make the following modifications:
vim Langchain-Chatchat/server/chat/chat.py
after line 48: callbacks.append(conversation_callback)

•  # Try to support langfuse
  

•  import os
  

•  langfuse_secret_key = os.environ.get('LANGFUSE_SECRET_KEY')
  

•  langfuse_public_key = os.environ.get('LANGFUSE_PUBLIC_KEY')
  

•  langfuse_host = os.environ.get('LANGFUSE_HOST')
  

•  if langfuse_secret_key and langfuse_public_key and langfuse_host :
  

•      from langfuse import Langfuse
  

•      from langfuse.callback import CallbackHandler
  

•      langfuse_handler = CallbackHandler()
  

•      callbacks.append(langfuse_handler)