Pass extra data to and from plugin

[jhcao23]

Hey @danny-avila , for non expert of Langchain, could you pls tell us how to pass extra data, besides user input string, to and from custom plugin. For example, in PluginsClient.js: we have this line this.result = await this.executor.call({ input, signal });. What if we want to pass extra data such as responseMessageId to the executor.call, and what if the StableDiffusionAPI.call wants to tell PluginsClient some extra data such as "this is not ready yet pls retrieve the image job 66777"? - Many thanks!

[danny-avila]

I will get back to you on this but what is your exact goal? It may help me understand what the best approach may be.

[jhcao23]

Lets say sdwebui or its variant supports async response. That means if it predicts some image may take 3min then it will tell the agent "hey dont wait. here is the job id you can check it again using the job status endpoint after 3min". So in this case StableDiffusionAPI needs the responseMessageId to tell another background job runner saying "fetch the image using this job id after 3min as well as update database or collections accordingly", and reply to agent saying "I have dispatched the delivery guy to pick up image after minutes. Here is the batch job id. You had better handle it for user".

[jhcao23]

btw, a typical SD-webui variant is RunPod Cloud which has a good pricing for "poor" startup LOL

[danny-avila]

Nice would love if you made a guide on setting that up.

So what you asked for is pretty complicated and would need some infrastructure added to the app.

however, you can use a structured tool with the functions agent to achieve a multi parameter function that the LLM can use. You could make an http call without awaiting it if it will be a while and count it good without waiting until the process is done, and send back an identifier right away

The tricky part is then how to reference it and it would have to depend on if the sdwebui api has a restful endpoint for checking if a certain image is done processing

[danny-avila]

I’m currently writing a guide on structuredTools so that it makes more sense