cloudflare[minor]: use native Cloudflare Workers AI binding #5287
Conversation
dosubot
bot
added
the
size:S
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
dosubot
bot
added
the
auto:improvement
![dosubot[bot]](https://avatars.githubusercontent.com/in/324583?s=60&v=4){kind=small}
| @@ -38,12 +38,11 @@ | |||
| "author": "LangChain", | |||
There was a problem hiding this comment.
Hey there! I noticed that this PR includes a change in dependencies, specifically the removal of "@cloudflare/ai" as a hard dependency and an update to the version of "@cloudflare/workers-types" as a dev dependency. I'm flagging this for your review to ensure it aligns with our dependency management strategy. Thanks!
| "@langchain/core": "~0.1", | ||
| "uuid": "^9.0.1" | ||
| }, | ||
| "devDependencies": { | ||
| "@cloudflare/workers-types": "^4.20231218.0", | ||
| "@cloudflare/workers-types": "^4.20240502.0", |
There was a problem hiding this comment.
If we're replacing @cloudflare/ai with this, it needs to be moved out of dev deps and into standard deps
There was a problem hiding this comment.
workers-types only provides the types for the Ai binding - there's nothing runtime that would need this to be a dependency and not a devDependency. The runtime for the AI binding is handled entirely in workerd via wrangler.
| @@ -14,7 +13,7 @@ type AiTextEmbeddingsOutput = { | |||
|
|
|||
| export interface CloudflareWorkersAIEmbeddingsParams extends EmbeddingsParams { | |||
| /** Binding */ | |||
| binding: Fetcher; | |||
| binding: Ai; | |||
There was a problem hiding this comment.
This will be a breaking change. Instead of this we should deprecate the binding input, and make it optional. With that we can add an additional (also optional) arg which is typed with the new Ai type.
There was a problem hiding this comment.
Hmm, good thoughts. The confusion here might come from the fact that the new AI binding is more of a binding than the old one (the old one is often referred to as a "wrapped binding" in the cf dev ecosystem), so I don't have a good name for this.
Do you have a suggestion for what this could be called? Or would it make sense to just cut a breaking change given how this is in 0.0.x versions right now?
There was a problem hiding this comment.
Yes, let's make this a minor version.
There was a problem hiding this comment.
Would this also break people using this outside Cloudflare Workers?
We'll also want to update the docs but I can take care of that.
There was a problem hiding this comment.
The external usage outside of Cloudflare Workers seems to be in https://github.com/Cherry/langchainjs/blob/4b9192176440b4acab9eac60a140200640d8bee2/libs/langchain-cloudflare/src/chat_models.ts#L78, which hasn't changed. This update to the embeddings.ts via CloudflareWorkersAIEmbeddingsParams seems to have only ever been intended for use inside of Cloudflare Workers, since it requires a Fetcher as the binding, which is a Cloudflare thing, unless I'm missing something?
Cherry
changed the title
|
Hey so just got wind that the Rest API methods will be available in the main SDK: https://www.npmjs.com/package/cloudflare We should wait to land this until that is generally ready |
|
The general Updating https://github.com/Cherry/langchainjs/blob/4b9192176440b4acab9eac60a140200640d8bee2/libs/langchain-cloudflare/src/chat_models.ts#L78 to use the official SDK once the AI endpoints are available there makes sense, but https://github.com/Cherry/langchainjs/blob/fix/workers-ai-native-binding/libs/langchain-cloudflare/src/embeddings.ts seems to be intended for use only within a Worker. |
|
Yeah for sure but ideally we don't lose functionality for people using it outside workers - I've done that myself in demos. It should accept either a binding or a config object/client IMO. |
|
Agreed that would be ideal, but it doesn't look like it does today: https://github.com/langchain-ai/langchainjs/blob/main/libs/langchain-cloudflare/src/embeddings.ts. No functionality here will be lost, so perhaps that improvement would be better served in another PR? |
As of 2024-04-11, there's no need to use the
@cloudflare/aipackage and it's deprecated:https://developers.cloudflare.com/workers-ai/changelog/
This swaps the code over to use the native binding directly, and updates
@cloudflare/workers-typeswhich now ships theAitype. No other code changes are needed.