Replies: 1 comment 13 replies
-
|
To address the issues you're encountering with the ConversationalRetrievalChain and FAISS retriever, consider the following steps:
These steps should help in resolving the issues with long processing times and errors related to exceeding token limits. If further assistance is needed, please provide more specific details about the problems you're facing.
|
Beta Was this translation helpful? Give feedback.
-
|
In the example, looks like you imported |
Beta Was this translation helpful? Give feedback.
-
|
Also I noticed that the verbose out put of llm_chain is in |
Beta Was this translation helpful? Give feedback.
-
|
To resolve the Here is the relevant part of the code that sets the @root_validator
def set_text(cls, values: Dict[str, Any]) -> Dict[str, Any]:
"""Set the text attribute to be the contents of the message."""
try:
text = ""
if isinstance(values["message"].content, str):
text = values["message"].content
# HACK: Assumes text in content blocks in OpenAI format.
# Uses first text block.
elif isinstance(values["message"].content, list):
for block in values["message"].content:
if isinstance(block, str):
text = block
break
elif isinstance(block, dict) and "text" in block:
text = block["text"]
break
else:
pass
else:
pass
values["text"] = text
except (KeyError, AttributeError) as e:
raise ValueError("Error while initializing ChatGeneration") from e
return valuesTo handle the verbose output of Here is an example of how you might create a from langchain_core.messages import BaseMessage
# Create a BaseMessage object with valid content
message = BaseMessage(content="This is a valid message content")
# Create a ChatGeneration object
chat_generation = ChatGeneration(message=message)
# The text attribute should now be correctly set
print(chat_generation.text) # Output: This is a valid message contentBy ensuring that the To handle the verbose output of from langchain_core.callbacks import CallbackManager, CallbackManagerForLLMRun
# Create a CallbackManager with verbose output enabled
callback_manager = CallbackManager(verbose=True)
# Create a run manager for the chain run
run_manager = CallbackManagerForLLMRun(callback_manager)
# Use the run manager in the LLMChain call
response = llm_chain._call(inputs, run_manager=run_manager)This will ensure that the verbose output is properly managed and displayed during the execution of the Additionally, the from typing import Any, Dict, Optional
class Generation:
"""A single text generation output."""
text: str
"""Generated text output."""
generation_info: Optional[Dict[str, Any]] = None
"""Raw response from the provider. May include things like the
reason for finishing or token log probabilities.
"""
type: str = "Generation"
"""Type is used exclusively for serialization purposes."""This class ensures that the For handling asynchronous calls and managing verbose output, you can use the async def _call_async(
self,
prompt: str,
stop: Optional[List[str]] = None,
callbacks: Callbacks = None,
*,
tags: Optional[List[str]] = None,
metadata: Optional[Dict[str, Any]] = None,
**kwargs: Any,
) -> str:
"""Check Cache and run the LLM on the given prompt and input."""
result = await self.agenerate(
[prompt],
stop=stop,
callbacks=callbacks,
tags=tags,
metadata=metadata,
**kwargs,
)
return result.generations[0][0].textThis method ensures that the verbose output is properly managed and displayed during the execution of the By following these steps, you should be able to resolve the
|
Beta Was this translation helpful? Give feedback.
-
|
Is there anything I can change in my code？ @dosu |
Beta Was this translation helpful? Give feedback.
-
|
To resolve the Here is an example of how you might initialize a from langchain_core.messages import BaseMessage
# Example message with string content
message = BaseMessage(content="This is a test message.")
# Initialize ChatGeneration with the message
chat_generation = ChatGeneration(message=message)
print(chat_generation.text) # Output: This is a test message.If the # Example message with list content
message = BaseMessage(content=["First block of text.", {"text": "Second block of text."}])
# Initialize ChatGeneration with the message
chat_generation = ChatGeneration(message=message)
print(chat_generation.text) # Output: First block of text.By ensuring the Handling Large Context LengthsTo handle large context lengths and avoid the
Here is an example of how to implement these strategies: from langchain_core.prompts import PromptTemplate
from langchain.memory.token_buffer import ConversationTokenBufferMemory
from langchain.chains import LLMChain
from langchain_core.messages import BaseMessage
from langchain_core.outputs import ChatGeneration
# Define the prompt template
template = (
"Combine the chat history and follow up question into "
"a standalone question. Chat History: {chat_history}"
"Follow up question: {question}"
)
prompt = PromptTemplate.from_template(template)
# Initialize the language model
llm = Ollama(base_url='http://localhost:11434', model="llama2:13b", num_gpu=1, num_thread=4)
# Create the LLM chain
llm_chain = LLMChain(llm=llm, prompt=prompt, verbose=True)
# Create the conversation memory with a token limit
memory = ConversationTokenBufferMemory(
llm=llm,
max_token_limit=3500
)
# Function to prune the buffer
def prune_buffer(buffer, max_token_limit):
curr_buffer_length = llm.get_num_tokens_from_messages(buffer)
while curr_buffer_length > max_token_limit:
buffer.pop(0)
curr_buffer_length = llm.get_num_tokens_from_messages(buffer)
return buffer
# Example usage
chat_history = [
BaseMessage(content="Hello, how can I help you today?"),
BaseMessage(content="I need information about the latest technology trends.")
]
# Prune the buffer if it exceeds the token limit
chat_history = prune_buffer(chat_history, 3500)
# Create a ChatGeneration object
message = BaseMessage(content="What are the latest technology trends?")
chat_generation = ChatGeneration(message=message)
print(chat_generation.text) # Output: What are the latest technology trends?By following these steps, you can handle large context lengths and ensure that the
|
Beta Was this translation helpful? Give feedback.
-
Checked other resources
Commit to Help
Example Code
Description
The chain runs for a long period of time but it's not generating answers. If I reduced the number of context length, I would get the following error:
number of tokens (32278) exceeded maximum context length (3500).The csv I'm using has little over 100 columnsSystem Info
langchain==0.1.20
langchain-community==0.0.38
langchain-core==0.1.52
langchain-text-splitters==0.0.1
Beta Was this translation helpful? Give feedback.
All reactions