Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

langchain[major],community[major],core[minor]: LangChain 0.2 #5204

Merged
merged 63 commits into from
May 13, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
63 commits
Select commit Hold shift + click to select a range
444d2f4
core[patch],langchain[major]: Move indexing API and document loader b…
jacoblee93 Apr 24, 2024
985bace
Flip community and LangChain dep (#5207)
jacoblee93 Apr 26, 2024
99a63a2
langchain[patch],scripts[patch]: Adds dynamic import map for serializ…
jacoblee93 Apr 26, 2024
99cc361
Merge
jacoblee93 Apr 26, 2024
5b80ac1
Move translators to new endpoints
jacoblee93 Apr 26, 2024
015e3c9
Publish as rc
jacoblee93 Apr 27, 2024
038998a
merge main
bracesproul May 2, 2024
22e577f
moved structured query to proper location
bracesproul May 2, 2024
0b9c1a8
fix load for deps
bracesproul May 2, 2024
df9851f
fix anthropic/community deps on each other
bracesproul May 2, 2024
dac8916
chore: lint files
bracesproul May 2, 2024
be94e27
cr
bracesproul May 2, 2024
fd6211e
update rcs
bracesproul May 2, 2024
8c41320
cr
bracesproul May 3, 2024
e33a50b
add back schema/query_constructor
bracesproul May 3, 2024
fffc43a
yarn
bracesproul May 3, 2024
0bd3c6d
cr
bracesproul May 3, 2024
bd9a2da
drop dynamic_import_map.ts
bracesproul May 3, 2024
db9d58d
fix tests:)
bracesproul May 3, 2024
4f5e0bd
fixes'
bracesproul May 3, 2024
2eb8e97
merge main
bracesproul May 3, 2024
aa7a140
update build
bracesproul May 3, 2024
5278546
format
bracesproul May 3, 2024
1334e94
fix cohere dep
bracesproul May 3, 2024
4b8e66a
drop bad
bracesproul May 3, 2024
7a76679
try building lc after
bracesproul May 3, 2024
ba10e98
mayb?
bracesproul May 3, 2024
bb3d20f
cr
bracesproul May 3, 2024
8b5f05f
add cohere key as secret
bracesproul May 3, 2024
a8db6e3
yarn why
bracesproul May 3, 2024
8becade
copy cohere pkg into export tests
bracesproul May 3, 2024
676bade
build cohere in lc
bracesproul May 3, 2024
24a9eaf
add cohere in dcope
bracesproul May 3, 2024
6ad190a
hehe mayb
bracesproul May 3, 2024
4ab0a0d
revert dep range tests changes
bracesproul May 3, 2024
faff833
cr
bracesproul May 3, 2024
7c0f7fc
build community too
bracesproul May 3, 2024
e73ec63
fix build
bracesproul May 3, 2024
210b998
add back tools
bracesproul May 3, 2024
60f5442
actually add back tools
bracesproul May 3, 2024
a947948
deprecated docs
bracesproul May 3, 2024
34aa858
add back non deprecated entrypoint
bracesproul May 3, 2024
ae2207a
added rest of tools
bracesproul May 3, 2024
2d7070f
add missing entrypoint
bracesproul May 3, 2024
cbf64f2
improvements
bracesproul May 3, 2024
4704335
fixes
bracesproul May 3, 2024
f834379
update examples
bracesproul May 3, 2024
fae586f
lint n fix
bracesproul May 3, 2024
1157f16
cr
bracesproul May 6, 2024
60fd49d
fix docs support
bracesproul May 6, 2024
3f12d69
chore: lint files
bracesproul May 6, 2024
74e2af1
chore: lint files
bracesproul May 6, 2024
5971c0a
did I fix it
bracesproul May 8, 2024
07a4226
merge main
bracesproul May 8, 2024
c4a840e
yer
bracesproul May 8, 2024
e475de3
bump all pkgs to 0.2
bracesproul May 8, 2024
37511d1
what
bracesproul May 8, 2024
e0a89fc
update int pkgs to be any core v less than 0.3
bracesproul May 13, 2024
3c888c1
Merge branch 'main' into jacob/0.2
bracesproul May 13, 2024
d54f831
pin oai & textsplitters to 0.2 rc
bracesproul May 13, 2024
cba050d
cr
bracesproul May 13, 2024
12d3869
all on 0.2 rc
bracesproul May 13, 2024
3b58ae1
Merge branch 'main' into jacob/0.2
bracesproul May 13, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
8 changes: 4 additions & 4 deletions .github/workflows/build-examples.yml
Original file line number Diff line number Diff line change
Expand Up @@ -40,9 +40,9 @@ jobs:
run: yarn install --immutable
- name: Build Core
run: yarn turbo:command build --filter=@langchain/core
- name: Build all except LangChain & Examples
run: yarn turbo:command build --filter=!examples --filter=!langchain --filter=!core_docs --filter=!api_refs --filter=!create-langchain-integration
- name: Build LangChain
run: yarn turbo:command build --filter=langchain
- name: Build all except LangChain Community & Examples
run: yarn turbo:command build --filter=!examples --filter=!@langchain/community --filter=!core_docs --filter=!api_refs --filter=!create-langchain-integration
- name: Build LangChain Community
run: yarn turbo:command build --filter=@langchain/community
- name: Build examples
run: yarn turbo:command build --filter=examples
1 change: 1 addition & 0 deletions .github/workflows/compatibility.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ env:
PUPPETEER_SKIP_DOWNLOAD: "true"
PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD: "true"
NODE_VERSION: "18.x"
COHERE_API_KEY: ${{ secrets.COHERE_API_KEY }}

# Run a separate job for each check in the docker-compose file,
# so that they run in parallel instead of overwhelming the default 2 CPU runner.
Expand Down
24 changes: 21 additions & 3 deletions .github/workflows/test-exports.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,13 @@ jobs:
cache: "yarn"
- name: Install dependencies
run: yarn install --immutable
- name: Build

- name: Build "@langchain/community"
run: yarn workspace @langchain/community build
shell: bash
env:
SKIP_API_DOCS: true
- name: Build LangChain
run: yarn workspace langchain build
shell: bash
env:
Expand All @@ -53,7 +59,13 @@ jobs:
cache: "yarn"
- name: Install dependencies
run: yarn install --immutable
- name: Build

- name: Build "@langchain/community"
run: yarn workspace @langchain/community build
shell: bash
env:
SKIP_API_DOCS: true
- name: Build LangChain
run: yarn workspace langchain build
shell: bash
env:
Expand All @@ -73,7 +85,13 @@ jobs:
cache: "yarn"
- name: Install dependencies
run: yarn install --immutable
- name: Build

- name: Build "@langchain/community"
run: yarn workspace @langchain/community build
shell: bash
env:
SKIP_API_DOCS: true
- name: Build LangChain
run: yarn workspace langchain build
shell: bash
env:
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/unit-tests-langchain.yml
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,8 @@ jobs:
run: yarn run build --filter=langchain
- name: Test
run: yarn run test:unit:ci --filter=langchain --concurrency=1
env:
COHERE_API_KEY: ${{ secrets.COHERE_API_KEY }}
# run: yarn run test:unit:ci -- --coverage
# - name: Upload coverage reports to Codecov
# uses: codecov/codecov-action@v3
Expand Down
2 changes: 2 additions & 0 deletions dependency_range_tests/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ services:
environment:
PUPPETEER_SKIP_DOWNLOAD: "true"
PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD: "true"
COHERE_API_KEY: ${COHERE_API_KEY}
working_dir: /app
volumes:
- ../langchain:/langchain
Expand All @@ -15,6 +16,7 @@ services:
environment:
PUPPETEER_SKIP_DOWNLOAD: "true"
PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD: "true"
COHERE_API_KEY: ${COHERE_API_KEY}
working_dir: /app
volumes:
- ../langchain:/langchain
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ const semver = require("semver");

const currentPackageJson = JSON.parse(fs.readFileSync("./package.json"));

if (currentPackageJson.dependencies["@langchain/core"]) {
if (currentPackageJson.dependencies["@langchain/core"] && !currentPackageJson.dependencies["@langchain/core"].includes("rc")) {
const minVersion = semver.minVersion(
currentPackageJson.dependencies["@langchain/core"]
).version;
Expand All @@ -17,7 +17,7 @@ if (currentPackageJson.dependencies["@langchain/core"]) {
};
}

if (currentPackageJson.dependencies["@langchain/community"]) {
if (currentPackageJson.dependencies["@langchain/community"] && !currentPackageJson.dependencies["@langchain/community"].includes("rc")) {
const minVersion = semver.minVersion(
currentPackageJson.dependencies["@langchain/community"]
).version;
Expand Down
12 changes: 6 additions & 6 deletions docs/core_docs/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,10 @@ docs/expression_language/streaming.md
docs/expression_language/streaming.mdx
docs/use_cases/tool_use/human_in_the_loop.md
docs/use_cases/tool_use/human_in_the_loop.mdx
docs/use_cases/query_analysis/quickstart.md
docs/use_cases/query_analysis/quickstart.mdx
docs/use_cases/query_analysis/index.md
docs/use_cases/query_analysis/index.mdx
docs/use_cases/question_answering/streaming.md
docs/use_cases/question_answering/streaming.mdx
docs/use_cases/question_answering/sources.md
Expand All @@ -53,10 +57,6 @@ docs/use_cases/question_answering/citations.md
docs/use_cases/question_answering/citations.mdx
docs/use_cases/question_answering/chat_history.md
docs/use_cases/question_answering/chat_history.mdx
docs/use_cases/query_analysis/quickstart.md
docs/use_cases/query_analysis/quickstart.mdx
docs/use_cases/query_analysis/index.md
docs/use_cases/query_analysis/index.mdx
docs/use_cases/graph/semantic.md
docs/use_cases/graph/semantic.mdx
docs/use_cases/graph/quickstart.md
Expand Down Expand Up @@ -109,12 +109,12 @@ docs/use_cases/extraction/how_to/examples.md
docs/use_cases/extraction/how_to/examples.mdx
docs/modules/model_io/output_parsers/custom.md
docs/modules/model_io/output_parsers/custom.mdx
docs/modules/memory/chat_messages/custom.md
docs/modules/memory/chat_messages/custom.mdx
docs/modules/model_io/chat/response_metadata.md
docs/modules/model_io/chat/response_metadata.mdx
docs/modules/model_io/chat/function_calling.md
docs/modules/model_io/chat/function_calling.mdx
docs/modules/memory/chat_messages/custom.md
docs/modules/memory/chat_messages/custom.mdx
docs/modules/data_connection/vectorstores/custom.md
docs/modules/data_connection/vectorstores/custom.mdx
docs/modules/agents/agent_types/tool_calling.md
Expand Down
21 changes: 18 additions & 3 deletions docs/core_docs/docs/integrations/chat/ni_bittensor.mdx
Original file line number Diff line number Diff line change
@@ -1,14 +1,29 @@
---
sidebar_label: NIBittensorChatModel
sidebar_class_name: hidden
---

# NIBittensorChatModel

:::warning
This module has been deprecated and is no longer supported. The documentation below will not work in versions 0.2.0 or later.
:::

LangChain.js offers experimental support for Neural Internet's Bittensor chat models.

Here's an example:

import CodeBlock from "@theme/CodeBlock";
import NIBittensorChatModelExample from "@examples/models/chat/integration_bittensor.ts";
```typescript
import { NIBittensorChatModel } from "langchain/experimental/chat_models/bittensor";
import { HumanMessage } from "@langchain/core/messages";

<CodeBlock language="typescript">{NIBittensorChatModelExample}</CodeBlock>
const chat = new NIBittensorChatModel();
const message = new HumanMessage("What is bittensor?");
const res = await chat.invoke([message]);
console.log({ res });
/*
{
res: "\nBittensor is opensource protocol..."
}
*/
```
10 changes: 0 additions & 10 deletions docs/core_docs/docs/integrations/llms/fake.mdx

This file was deleted.

40 changes: 19 additions & 21 deletions docs/core_docs/docs/integrations/llms/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -18,24 +18,22 @@ All LLMs implement the Runnable interface, which comes with default implementati

Each LLM integration can optionally provide native implementations for invoke, streaming or batch, which, for providers that support it, can be more efficient. The table shows, for each integration, which features have been implemented with native support.

| Model | Invoke | Stream | Batch |
| :-------------------- | :----: | :----: | :---: |
| AI21 | ✅ | ❌ | ✅ |
| AlephAlpha | ✅ | ❌ | ✅ |
| AzureOpenAI | ✅ | ✅ | ✅ |
| CloudflareWorkersAI | ✅ | ✅ | ✅ |
| Cohere | ✅ | ❌ | ✅ |
| Fireworks | ✅ | ✅ | ✅ |
| GooglePaLM | ✅ | ❌ | ✅ |
| HuggingFaceInference | ✅ | ❌ | ✅ |
| LlamaCpp | ✅ | ✅ | ✅ |
| Ollama | ✅ | ✅ | ✅ |
| OpenAI | ✅ | ✅ | ✅ |
| OpenAIChat | ✅ | ✅ | ✅ |
| PromptLayerOpenAI | ✅ | ✅ | ✅ |
| PromptLayerOpenAIChat | ✅ | ✅ | ✅ |
| Portkey | ✅ | ✅ | ✅ |
| Replicate | ✅ | ❌ | ✅ |
| SageMakerEndpoint | ✅ | ✅ | ✅ |
| Writer | ✅ | ❌ | ✅ |
| YandexGPT | ✅ | ❌ | ✅ |
| Model | Invoke | Stream | Batch |
| :------------------- | :----: | :----: | :---: |
| AI21 | ✅ | ❌ | ✅ |
| AlephAlpha | ✅ | ❌ | ✅ |
| AzureOpenAI | ✅ | ✅ | ✅ |
| CloudflareWorkersAI | ✅ | ✅ | ✅ |
| Cohere | ✅ | ❌ | ✅ |
| Fireworks | ✅ | ✅ | ✅ |
| GooglePaLM | ✅ | ❌ | ✅ |
| HuggingFaceInference | ✅ | ❌ | ✅ |
| LlamaCpp | ✅ | ✅ | ✅ |
| Ollama | ✅ | ✅ | ✅ |
| OpenAI | ✅ | ✅ | ✅ |
| OpenAIChat | ✅ | ✅ | ✅ |
| Portkey | ✅ | ✅ | ✅ |
| Replicate | ✅ | ❌ | ✅ |
| SageMakerEndpoint | ✅ | ✅ | ✅ |
| Writer | ✅ | ❌ | ✅ |
| YandexGPT | ✅ | ❌ | ✅ |
25 changes: 22 additions & 3 deletions docs/core_docs/docs/integrations/llms/ni_bittensor.mdx
Original file line number Diff line number Diff line change
@@ -1,10 +1,29 @@
---
sidebar_class_name: hidden
---

# NIBittensor

:::warning
This module has been deprecated and is no longer supported. The documentation below will not work in versions 0.2.0 or later.
:::

LangChain.js offers experimental support for Neural Internet's Bittensor LLM models.

Here's an example:

import CodeBlock from "@theme/CodeBlock";
import NIBittensorExample from "@examples/models/llm/ni_bittensor.ts";
```typescript
import { NIBittensorLLM } from "langchain/experimental/llms/bittensor";

const model = new NIBittensorLLM();

const res = await model.invoke(`What is Bittensor?`);

console.log({ res });

<CodeBlock language="typescript">{NIBittensorExample}</CodeBlock>
/*
{
res: "\nBittensor is opensource protocol..."
}
*/
```
8 changes: 8 additions & 0 deletions docs/core_docs/docs/integrations/llms/prompt_layer_openai.mdx
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
---
sidebar_class_name: hidden
---

# PromptLayer OpenAI

:::warning
This module has been deprecated and is no longer supported. The documentation below will not work in versions 0.2.0 or later.
:::

LangChain integrates with PromptLayer for logging and debugging prompts and responses. To add support for PromptLayer:

1. Create a PromptLayer account here: [https://promptlayer.com](https://promptlayer.com).
Expand Down
8 changes: 0 additions & 8 deletions docs/core_docs/docs/integrations/platforms/openai.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -48,14 +48,6 @@ See a [usage example](/docs/integrations/text_embedding/openai)
import { OpenAIEmbeddings } from "@langchain/openai";
```

## Retriever

See a [usage example](/docs/integrations/retrievers/chatgpt-retriever-plugin).

```typescript
import { ChatGPTPluginRetriever } from "langchain/retrievers/remote";
```

## Chain

```typescript
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,27 @@ hide_table_of_contents: true

# ChatGPT Plugin Retriever

:::warning
This module has been deprecated and is no longer supported. The documentation below will not work in versions 0.2.0 or later.
:::

This example shows how to use the ChatGPT Retriever Plugin within LangChain.

To set up the ChatGPT Retriever Plugin, please follow instructions [here](https://github.com/openai/chatgpt-retrieval-plugin).

## Usage

import CodeBlock from "@theme/CodeBlock";
import Example from "@examples/retrievers/chatgpt-plugin.ts";
```typescript
import { ChatGPTPluginRetriever } from "langchain/retrievers/remote";

const retriever = new ChatGPTPluginRetriever({
url: "http://0.0.0.0:8000",
auth: {
bearer: "super-secret-jwt-token-with-at-least-32-characters-long",
},
});

const docs = await retriever.invoke("hello world");

<CodeBlock language="typescript">{Example}</CodeBlock>
console.log(docs);
```
41 changes: 37 additions & 4 deletions docs/core_docs/docs/integrations/tools/zapier_agent.mdx
Original file line number Diff line number Diff line change
@@ -1,7 +1,13 @@
import CodeBlock from "@theme/CodeBlock";
---
sidebar_class_name: hidden
---

# Agent with Zapier NLA Integration

:::warning
This module has been deprecated and is no longer supported. The documentation below will not work in versions 0.2.0 or later.
:::

Full docs here: https://nla.zapier.com/start/

**Zapier Natural Language Actions** gives you access to the 5k+ apps and 20k+ actions on Zapier's platform through a natural language API interface.
Expand All @@ -22,8 +28,6 @@ Review [auth docs](https://nla.zapier.com/docs/authentication/) for more details

The example below demonstrates how to use the Zapier integration as an Agent:

import Example from "@examples/agents/zapier_mrkl.ts";

import IntegrationInstallTooltip from "@mdx_components/integration_install_tooltip.mdx";

<IntegrationInstallTooltip></IntegrationInstallTooltip>
Expand All @@ -32,4 +36,33 @@ import IntegrationInstallTooltip from "@mdx_components/integration_install_toolt
npm install @langchain/openai
```

<CodeBlock language="typescript">{Example}</CodeBlock>
```typescript
import { OpenAI } from "@langchain/openai";
import { ZapierNLAWrapper } from "langchain/tools";
import {
initializeAgentExecutorWithOptions,
ZapierToolKit,
} from "langchain/agents";

const model = new OpenAI({ temperature: 0 });
const zapier = new ZapierNLAWrapper();
const toolkit = await ZapierToolKit.fromZapierNLAWrapper(zapier);

const executor = await initializeAgentExecutorWithOptions(
toolkit.tools,
model,
{
agentType: "zero-shot-react-description",
verbose: true,
}
);
console.log("Loaded agent.");

const input = `Summarize the last email I received regarding Silicon Valley Bank. Send the summary to the #test-zapier Slack channel.`;

console.log(`Executing with input "${input}"...`);

const result = await executor.invoke({ input });

console.log(`Got output ${result.output}`);
```