Need help to convert my single-agent project into a multi-agent one

[MOUNAJEDK]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

import os
from dotenv import load_dotenv
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain.agents import AgentExecutor, create_openai_tools_agent, Tool
from langchain_core.runnables.history import RunnableWithMessageHistory
from langchain_mongodb.chat_message_histories import MongoDBChatMessageHistory
from serpapi import GoogleSearch

# Load environment variables for API keys
load_dotenv()

# SerpAPI and MongoDB configuration
serpapi_key = os.getenv("SERPAPI_API_KEY")
mongo_connection_string = "mongodb://localhost:27017"
database_name = "chatbot_db"
collection_name = "chat_histories"

# Define the function that will use SerpApi to perform searches
def perform_serpapi_search(query):
    params = {
        "engine": "google",
        "q": query,
        "api_key": serpapi_key,
        "num": 5,
    }
    search = GoogleSearch(params)
    results = search.get_dict()
    return results

# Create the SerpApi tool to pass to an agent
serpapi_tool = Tool(
    name="serpapi_search",
    description="Performs Google searches using SerpApi.",
    func=perform_serpapi_search
)

# Setup the ChatOpenAI model for conversational interactions
chat = ChatOpenAI(
    model='gpt-3.5-turbo-1106',
    temperature=0
)

# Define a comprehensive prompt template for handling game recommendations
game_recommendation_prompt = ChatPromptTemplate.from_messages(
    [
        ("system", """
                    You are a sophisticated AI trained to recommend video games. Your tasks include:

                    - Provide game suggestions similar to ones the user enjoys or mentions, covering various genres and platforms.
                    - Recommend games based on specific genres or mentioned developers/publishers.
                    - Identify and suggest top-trending and highly rated video games, including acclaimed titles from specific time periods.
                    - Tailor recommendations according to user-defined preferences, such as complexity, time investment, and progression style.
                    - Recommend games suitable for specified platforms (e.g., PlayStation, Xbox, PC, Switch) or fitting certain age ratings (e.g., E, T, M).
                    - Replace played or unappealing games with suitable alternatives.
                    - For each recommended game, provide: title, brief description, genre, platform, developer, publisher, release date, Metacritic score (if available), and purchase links from digital storefronts.
                    - Politely request more specific information for ambiguous queries.
                    - Guide users back to gaming-related topics for unrelated queries.
                    - Maintain a friendly and engaging tone throughout interactions.
                    - Utilize the SerpAPI search tool for up-to-date and accurate recommendations in each response to user queries. (VERY IMPORTANT!!!)

                    Ensure clarity, conciseness, and engagement in your responses to enhance the user experience.
                    """),
        MessagesPlaceholder(variable_name="chat_history"),
        MessagesPlaceholder(variable_name="agent_scratchpad"),
        ("human", "{input}"),
    ]
)

# Setup tools for agent
tools = [serpapi_tool]

# Create an OpenAI tools agent for handling game recommendations
game_recommendation_agent = create_openai_tools_agent(chat, tools, game_recommendation_prompt)

# Setup the agent executor for managing operations
agent_executor = AgentExecutor(agent=game_recommendation_agent, tools=tools, verbose=True)

# Function to manage MongoDB-based message history for each session
def get_message_history(session_id):
    return MongoDBChatMessageHistory(
        session_id=session_id,
        connection_string=mongo_connection_string,
        database_name=database_name,
        collection_name=collection_name,
    )

# Function to handle user queries
def handle_user_query(session_id, user_input):
    """
    Process user queries by wrapping the executor with RunnableWithMessageHistory
    which processes various types of game recommendation requests and manages user interaction,
    maintaining a history of the conversation in MongoDB.
    """
    history_manager = RunnableWithMessageHistory(
        agent_executor,
        lambda session_id: get_message_history(session_id),
        input_messages_key="input",
        output_messages_key="output",
        history_messages_key="chat_history",
    )

    # Execute the query with history management
    response = history_manager.invoke(
        {"input": user_input},
        {"configurable": {"session_id": session_id}}
    )
    
    return response['output']

def main():
    session_id = "unique_user_session_id"  # This should be uniquely generated for each user session
    print("Welcome to the Game Recommendation Chatbot!")
    while True:
        user_input = input("You: ")
        if user_input.lower() == 'exit':
            print("Exiting chatbot...")
            break
        response = handle_user_query(session_id, user_input)
        print("Bot:", response)

if __name__ == "__main__":
    main()

Description

Hey guys! So for context, I'm trying to develop a simple chatbot the offers personalized video games recommendations based on user input, by searching the internet for the top results and then use them as an answer to the user. Initially, I started this using one single agent, but as more ideas came into my mind, I think sticking with only one agent and try to implement those into code my result in some issues, specifically when it comes to the number of tokens. So I've decided instead to leverage LangGraph in order to adopt the multi-agent way and thus some myself from some trouble. Here is what I was thinking about when it comes to the agents I have thought of and their objective:

1. Input Agent (Agent 1): This agent receives the initial user input, interprets the user's query, and dispatches tasks to other specialized agents.
2. Search Agent (Agent 2): Receives tasks from Agent 1 to perform initial searches for game titles.
3. Details Agent (Agent 3): Fetches detailed information for each game identified by Agent 2.
4. Posters and Trailers Agents (Agent 4 and 5): Responsible for fetching official posters and official video trailers for the games identified.

Then I was thinking of sending all the details from Agent 2, 3, 4 and 5 to a core agent responsible for formatting a response based on them and then display them to the user.

Problem is, so far, I keep failing in my attempt to move from LangChain to LangGraph whilst trying to implement these ideas into code.

Can anyone please help? I would really, really, appreciate some help with the implementation of this.

System Info

Python 3.12

[dosubot[bot]]

To evolve your chatbot from a single-agent to a multi-agent system using LangGraph, you'll need to focus on defining and implementing each agent, setting up their interactions, and orchestrating these interactions to process user inputs effectively. Here's a streamlined approach:

1. Define and Implement Agents: Start by creating specialized agents for different tasks (e.g., genre analysis, user preferences) using create_structured_chat_agent. Each agent should have its own prompt and tools tailored to its specific role.

2. Agent Interactions with StateGraph: Use StateGraph to map out the interactions between your agents. Create nodes for each agent and define edges that dictate how information flows from one agent to another, ensuring a coherent process from user input to final recommendation.

3. Workflow Configuration: Within your StateGraph, specify the start node and how data should be merged or selected from different agents to form the final output. This setup allows for a modular and flexible system that can easily be adjusted or expanded.

4. Execution: With the graph configured, execute it with user inputs to generate personalized recommendations. This execution step ties together the work of all individual agents into a cohesive output.

By modularizing your chatbot's functionality, you make it easier to maintain and extend. Each agent can be independently developed, tested, and integrated, providing a clear structure for their interactions and the overall flow of information.

Details

Sources

• Allow agent to choose its toolset
• how to improve the perfromance of agents to get better responses from the local model like gpt4all
• libs/langchain/langchain/agents/structured_chat/base.py
• cookbook/langgraph_crag.ipynb
• cookbook/langgraph_self_rag.ipynb
• cookbook/langgraph_agentic_rag.ipynb
• docs/docs/modules/agents/agent_types/structured_chat.ipynb
• docs/docs/modules/agents/how_to/agent_structured.ipynb
• libs/partners/cohere/langchain_cohere/react_multi_hop/agent.py
• libs/langchain/langchain/agents/init.py
• libs/core/langchain_core/runnables/graph_png.py

About Dosu
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.

[MOUNAJEDK]

@dosu
I've update my code to the following trying to implement my idea:
`
import os
from dotenv import load_dotenv
from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage, BaseMessage
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain.agents import create_openai_tools_agent, Tool
from langgraph.graph import StateGraph, END
from typing import Annotated, Sequence, TypedDict
import operator
from serpapi import GoogleSearch

Load environment variables for API keys

load_dotenv()

Configuration

serpapi_key = os.getenv("SERPAPI_API_KEY")

Define tools using SerpAPI

def google_search(query):
params = {
"engine": "google",
"q": query,
"api_key": serpapi_key,
"num": 5
}
search = GoogleSearch(params)
results = search.get_dict()
return results['organic_results']

def youtube_search(query):
params = {
"engine": "youtube",
"search_query": query,
"api_key": serpapi_key,
"num": 1
}
search = GoogleSearch(params)
results = search.get_dict()
return results['video_results']

def images_search(query):
params = {
"engine": "google_images",
"q": query,
"api_key": serpapi_key,
"num": 1
}
search = GoogleSearch(params)
results = search.get_dict()
return results['images_results']

Define tools

search_tool = Tool(name="Google Search", description="Performs Google searches.", func=google_search)
youtube_tool = Tool(name="YouTube Search", description="Searches for YouTube videos.", func=youtube_search)
images_tool = Tool(name="Images Search", description="Searches for images on Google Images.", func=images_search)

Setup the ChatOpenAI model for conversational interactions

chat_model = ChatOpenAI(model='gpt-3.5-turbo-1106', temperature=0)

Define agent prompts

input_agent_prompt = ChatPromptTemplate.from_messages([
("system", "You are the initial contact point for users. Your role is to understand the user's query and determine the type of request. Focus on identifying whether the user is asking for game recommendations, seeking information about a specific game, or needing help with gaming platforms."),
MessagesPlaceholder(variable_name="messages"),
MessagesPlaceholder(variable_name="agent_scratchpad"),

("human", "{input}")

])

search_agent_prompt = ChatPromptTemplate.from_messages([
("system", "Your main task is to search for game titles that best match the user's interest. Use the details from the Input Agent to guide your search. Provide a list of relevant game titles."),
MessagesPlaceholder(variable_name="messages"),
MessagesPlaceholder(variable_name="agent_scratchpad"),
("human", "{input}")
])

details_agent_prompt = ChatPromptTemplate.from_messages([
("system", "Retrieve detailed information for each game identified by the Search Agent. Focus on obtaining the game's description, genre, platform availability, developer, publisher, release date, Metacritic score if available and links to digital stores that sell the game."),
MessagesPlaceholder(variable_name="messages"),
MessagesPlaceholder(variable_name="agent_scratchpad"),
("human", "{input}")
])

posters_agent_prompt = ChatPromptTemplate.from_messages([
("system", "Your responsibility is to fetch the official posters for the games provided by the Search Agent. Ensure the images are high quality and relevant."),
MessagesPlaceholder(variable_name="messages"),
MessagesPlaceholder(variable_name="agent_scratchpad"),
("human", "{input}")
])

trailers_agent_prompt = ChatPromptTemplate.from_messages([
("system", "Obtain the official game trailers for the titles identified by the Search Agent. Ensure that the trailers are current and of high quality."),
MessagesPlaceholder(variable_name="messages"),
MessagesPlaceholder(variable_name="agent_scratchpad"),
("human", "{input}")
])

recommendation_agent_prompt = ChatPromptTemplate.from_messages([
("system", "You are responsible for compiling the outputs from all other agents into a cohesive and well-formatted response. Synthesize the game details, images, and trailers into a compelling presentation of game recommendations."),
MessagesPlaceholder(variable_name="messages"),
MessagesPlaceholder(variable_name="agent_scratchpad"),
("human", "{input}")
])

Create agents using the prompts

input_agent = create_openai_tools_agent(chat_model, [], input_agent_prompt)
search_agent = create_openai_tools_agent(chat_model, [search_tool], search_agent_prompt)
details_agent = create_openai_tools_agent(chat_model, [search_tool], details_agent_prompt)
posters_agent = create_openai_tools_agent(chat_model, [images_tool], posters_agent_prompt)
trailers_agent = create_openai_tools_agent(chat_model, [youtube_tool], trailers_agent_prompt)
recommendation_agent = create_openai_tools_agent(chat_model, [], recommendation_agent_prompt)

State definition for agents

class AgentState(TypedDict):
messages: Annotated[Sequence[BaseMessage], operator.add]
intermediate_steps: Annotated[Sequence[BaseMessage], operator.add]
agent_scratchpad: Annotated[Sequence[BaseMessage], operator.add]
input: str

Graph setup

state_graph = StateGraph(schema=AgentState)

Add nodes for each agent

state_graph.add_node("input_agent", input_agent)
state_graph.add_node("search_agent", search_agent)
state_graph.add_node("details_agent", details_agent)
state_graph.add_node("posters_agent", posters_agent)
state_graph.add_node("trailers_agent", trailers_agent)
state_graph.add_node("recommendation_agent", recommendation_agent)

Define edges to flow between agents

state_graph.add_edge("input_agent", "search_agent")
state_graph.add_edge("search_agent", "details_agent")
state_graph.add_edge("details_agent", "posters_agent")
state_graph.add_edge("posters_agent", "trailers_agent")
state_graph.add_edge("trailers_agent", "recommendation_agent")
state_graph.add_edge("recommendation_agent", END)

Set the entry point to start the agent workflow

state_graph.set_entry_point("input_agent")

Complete the graph

app = state_graph.compile()

def main():
print("Welcome to the Game Recommendation Chatbot!")
while True:
user_input = input("You: ")
if user_input.lower() == 'exit':
print("Exiting chatbot...")
break
try:
# Initialize the state with the necessary structures
state = {
"messages": [HumanMessage(content=user_input)],
"agent_scratchpad": [], # ensure this is always a list
"intermediate_steps": [], # ensure this is always initialized as a list
"input": user_input # this is your actual input string
}
response = app.invoke(state)
print("Bot:", response["messages"][-1].content)
except Exception as e:
print("An error occurred:", e)

if name == "main":
main()
`

Problem is, upon sending user input for the first time, I get this error: An error occurred: Error code: 400 - {'error': {'message': "[] is too short - 'tools'", 'type': 'invalid_request_error', 'param': None, 'code': None}}

[dosubot]

The error you're encountering, Error code: 400 - {'error': {'message': "[] is too short - 'tools'", 'type': 'invalid_request_error', 'param': None, 'code': None}}, suggests that there's an issue with how tools are being passed to the create_openai_tools_agent function. Specifically, it seems that the tools parameter is either empty or not being recognized as a valid sequence of BaseTool instances. This is crucial because each agent that requires tools must be provided with a non-empty list of these tools for proper operation.

To address this issue, you should:

1. Verify Tools Initialization: Ensure that each tool (e.g., search_tool, youtube_tool, images_tool) is correctly instantiated and that these instances are indeed being passed into the create_openai_tools_agent function where required. Each tool should be an instance of a class that inherits from BaseTool, properly initialized with the necessary parameters.

2. Check Tools Sequence: When you're passing the tools parameter to create_openai_tools_agent, make sure it's a sequence (like a list) that contains valid BaseTool instances. An empty list or a list not correctly recognized by the function could lead to the error you're seeing.

3. Debugging: Before the agent creation call, use print statements or a debugger to inspect the tools variable to ensure it's not empty and contains the expected BaseTool instances. This step will help you confirm that the tools are correctly set up and recognized by the system.

Here's a simplified example to ensure clarity on how tools should be initialized and passed:

# Example tool initialization and agent creation
from langchain.agents import create_openai_tools_agent, AgentExecutor
from langchain_core.tools import BaseTool

class ExampleTool(BaseTool):
    def __init__(self, name, description=""):
        super().__init__(name=name, description=description)

    def run(self, input):
        return input

# Assuming 'model' and 'template' are already defined
tools = [ExampleTool(name="ExampleTool", description="An example tool.")]
agent = create_openai_tools_agent(model, tools, template)
executor = AgentExecutor(agent=agent, tools=tools)

Ensure that for each agent requiring tools, the tools parameter is populated similarly to the example above. This approach should help you resolve the error and proceed with your multi-agent system implementation.

Details

About Dosu
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.

[MOUNAJEDK]

@dosu
But what in the scenario of creating an agent that I intend to not provide any tools to, is that possible? And if so, how can I do that? Or am I obliged to give to every agent out there a list of tools to use?

[dosubot]

Yes, it's entirely possible to create an agent that doesn't use any tools in your multi-agent system. When setting up such an agent with the create_openai_tools_agent function, you can pass an empty list for the tools parameter. This is common for agents whose tasks are purely based on interactions with the language model and do not require external data or functionalities that tools would provide.

Here's how you can do it for an agent that doesn't need any tools:

# Assuming 'chat_model' and 'input_agent_prompt' are already defined
input_agent = create_openai_tools_agent(chat_model, [], input_agent_prompt)

In this example, the empty list [] passed as the tools parameter signifies that the agent operates without any external tools. This setup is valid and useful for scenarios where the agent's functionality is self-contained within the capabilities of the language model itself.

This flexibility allows for a diverse range of agents within your system, some of which may interact with external APIs or services through tools, while others rely solely on the language model for processing and decision-making.

Details

About Dosu
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.