🤖

Hey @pacoccino! Great to see you back with another interesting idea. How's everything going?

Thank you for your feature request. Adding support for MLC-AI's Web-LLM to the LangChainJS framework could indeed enhance the functionality of applications developed with LangChain. However, integrating this feature would require significant changes to the current architecture of the LangChainJS codebase.

Currently, the execution of language models in the LangChainJS framework is handled by the BaseLanguageModel class, which provides several methods for executing language models, including generatePrompt, predict, and predictMessages. These methods are abstract and their actual implementation is in the subclasses of BaseLanguageModel.

To add support for Web-LLM, we would need to create a new subclass of BaseLanguageModel that implements these methods using the Web-LLM API. This would involve writing code to load the Web-LLM WASM module, initialize it with the appropriate language model, and then use it to execute the language model on the input text.

In addition, we would need to modify the Chains module to use this new subclass when the user wants to execute a language model in the browser. This would likely involve adding a new option to the Chains module's configuration that specifies whether to use the Web-LLM or the current method for executing language models.

Finally, we would need to thoroughly test this new feature to ensure that it works correctly and does not introduce any new bugs into the LangChainJS framework.

Unfortunately, as a technical support representative, I am unable to implement this feature myself. However, I would recommend creating a new issue on the LangChainJS GitHub repository detailing your feature request. This will allow the LangChainJS developers to evaluate your request and potentially implement it in a future update.

Please note that while I strive to provide accurate and up-to-date information, there may be a better solution that I was not able to find. If you have any further questions or need more information, please don't hesitate to ask.

Here is the link to the LangChainJS GitHub repository where you can create a new issue: LangChainJS GitHub

I hope this information is helpful. Thank you for your interest in the LangChainJS framework.

Best regards,
Dosu

Sources

• langchain/langchain.config.js
• langchain-core/src/language_models/base.ts

Yes this would be incredible!

I don't know much the codebase of langchain, does the suggestions of the bot are correct ?

Does this would "require significant changes to the current architecture" as the bot says, or it may be easy implementing this, as for HF's transformers.js ?

I can give it a try it there's not big architectures changes involved

@pacoccino Have you made significant effort towards this? I'm willing to pick this up.

I ended up adapting his PR on another repo here:

https://github.com/jacoblee93/fully-local-pdf-chatbot/blob/main/app/lib/chat_models/webllm.ts

Still a few rough edges but if you're down to refine it we'd welcome the addition!

Awesome. I'll take it on.

Hey! I tried running latest implementation that @jacoblee93 put together, but when compiling I am getting an import error for 'Engine' from the "@mlc-ai/web-llm" library from what looks like the webllm.ts file in the langchain-community library. The webllm.ts file appears to use 'webllm.Engine()' in initialization, yet when I looked into the "@mlc-ai/web-llm" library I notice they export a 'MLCEngine' class instead of 'Engine'. Is this a known issue/discrepancy with a resolution?

I think there was a breaking change in a recent WebLLM version a little while ago - what version did you install?

For the "@mlc-ai/web-llm" library, I tried a sweep from 0.2.38 thru 0.2.30, and several below - including some 0.1.x releases- but didn't find any change in the error. (I did the sweep for both "@langchain/community" 0.2.0 and 0.2.1 versions)