- Overview
- Current Features
- Running Modes
- Setup MongoDB Atlas
- Running Services
- Running Services (development)
Langtrace is an LLM tracing tool that consumes the data from Langchain and saves the data to MongoDB allowing for easy querying and visualization of the data through either the UI or MongoDB Atlas Charts.
It is built using Next.js, Node.js 20, Typescript and MongoDB Atlas.
- Ingests trace data from Langchain/Langsmith
- Handles feedback creation and updating
- UI to view trace data
- UI authentication providers (Github, Microsoft AD)
- MongoDB Trigger to add token usage data to the trace data
- Assumes all project names set in LANGCHAIN_PROJECT are URL safe, as they are used in the URL
- No need to register projects, they are automatically created when data is ingested
- No API key required for ingestion or querying
Langtrace can be run in two modes: Full Mode and Headless Mode. Full mode runs the UI and APIs together.
Headless mode runs the ingestion api only
This will start 3 services:
- UI
- Langtrace API (backend for the UI)
- Ingest Server
This will start 1 service:
- Ingest Server
It's assumed to view data you will use MongoDB Atlas Charts
- Create/Provision cluster in MongoDB Atlas
- Create a database in the cluster and set LANGTRACE_MONGODB_DB_NAME to the name of the database in the .env file in /server. Example langtrace
- Create a collection in the database and set LANGTRACE_TRACES_MONGODB_COLLECTION_NAME to the name of the collection in the .env file in /server. Example traces
- Create a readWrite user for the trace collection and setup a connection string for that user on LANGTRACE_INGEST_MONGODB_URI
- Create a readOnly user for the trace collection and setup a connection string for that user on LANGTRACE_API_MONGODB_URI
MongoDB Atlas required for this feature. If you are using MongoDB outside of Atlas you will need to handle the updates separately.
Example of a function to add token usage data to the trace data
in ./atlas/functions/add-token-usage.js
Create a trigger in MongoDB Atlas to run the function on the traces collection ( LANGTRACE_TRACES_MONGODB_COLLECTION_NAME)
-
In the MongoDB Atlas UI, navigate to Triggers
-
Create a new trigger
-
Select the cluster and database
-
Select the collection
-
Set Operation Type to Insert and Update
-
In advanced settings set the following:
- Match Expression:
{ "fullDocument.run_type": "llm", "fullDocument.end_time": { "$exists": true, "$ne": null }, "fullDocument.extra.tokens": { "$exists": false }, "fullDocument.outputs.generations": { "$exists": true, "$ne": null }, "fullDocument.inputs.messages": { "$exists": true, "$ne": null } }
- Project Expression:
{ "fullDocument.end_time": 1, "fullDocument.run_type": 1, "fullDocument.outputs.generations": 1, "fullDocument.extra.invocation_params.model": 1, "fullDocument.inputs.messages": 1, "ns.coll": 1, "documentKey._id": 1, "operationType": 1 }
-
Got to function and select the function you created and paste the code from the function file ./atlas/functions/add-token-usage.js
Assumes you have an existing project configured for defining you Atlas infrastructure
- Install the MongoDB Atlas CLI
- Copy the contents of ./atlas/functions/ into the functions folder in your project
- Copy the contents of ./atlas/trigger/ into the trigger folder in your project
- In the trigger file set the following fields
service-name
- the name of your servicedatabase
- the name of your database (if different from langtrace)collection
- the name of your collection (if different from traces) For further information on configuration see the [Trigger Configuration Files documentation] (https://www.mongodb.com/docs/atlas/app-services/reference/config/triggers/)
- Deploy changes
- MongoDB (some features like token usage currently depends on MongoDB Atlas) see Setup MongoDB
- Langchain application (see Langchain Configuration)
- Docker & Docker Compose
- In ./ui copy .env.example to .env and set the values
- In ./server copy .env.example to .env and set the values
Note: Langtrace is currently configured to use the
langsmith-sdk
.
- Add Langsmith dependency to your Langchain app
pip install langsmith
ornpm i langsmith
- Add the following VARs to your Langchain app
LANGCHAIN_TRACING_V2
- set totrue
LANGCHAIN_ENDPOINT
- the URL of your Langtrace APILANGCHAIN_PROJECT
- the name of your project
docker-compose up --build -f docker-compose.full.yml
docker-compose up --build -f docker-compose.headless.yml
- Node.js (tested on 20)
- MongoDB (some features like token usage currently depends on MongoDB Atlas) see Setup MongoDB
- Langchain application (see Langchain Configuration)
- In ./packages/ui copy .env.example to .env and set the values
- In ./packages/api copy .env.example to .env and set the values
- In ./packages/ingest copy .env.example to .env and set the values
Install the dependencies:
npm install
npm run dev --workspace=@langtrace/ui
This is the API that the UI uses to get data from MongoDB
npm run dev --workspace=@langtrace/api
This is the server that listens for data from Langchain
npm run dev --workspace=@langtrace/ingest