[PoC] Implement a state machine in the AiServices #861
Conversation
|
Hi @mariofusco, thanks a lot! I'll have a closer look after a long weekend, happy holidays! |
Consider this my Easter egg :) Enjoy your holidays. |
|
Hi @mariofusco, I like the idea of externalizing state transition logic, users should definitely be able to have total freedom there. A bit of context: I was considering having 2 options for implementing state machines with AI Services:
I will need more time to play with your proposal and other alternatives and see what will be a good fit for the option 2. But for now I think it is safe to go ahead with option 1 as it will be helpful anyway for many other use cases not involving state machines. |
As I wrote, I completely agree that 1. is the best option: on the short term it will enable the implementation of a few use cases, including the use of a completely external state machine as in my use case, and it could also be seen as a first step for the implementation of an internal state machine if we will decide that it is necessary. Since we are on the same page can we continue this discussion on the other pull request and check together what's missing there? I guess that the first thing to be implemented is the support to configure messages for each user/conversation, correct? Can we use a mechanism similar to what has been already developed for the chat memory for this or do you have any better idea? |
…rvices (#862) This pull request is a variation and simplification of what I proposed [here](#861). As anticipated there instead of making the AiServices aware of the concept of a state machine I simply added the possibility of making the system and user messages configurable in a programmatic way. Currently this pull request has the same limitation of the other one: lack of the possibility of configuring messages for each user/conversation and of support of tools, but I believe that both things could be added once we agree on the main direction.
For now this is only a PoC to start a discussion around the possibility of implementing a state machine inside the AiServces. This pull request is a complete rework of this other one and tries to put my proposal at work using the same use case that I implemented here.
In my opinion the original pull request was a bit inflexible, trying to model all the states and possible transactions among them inside the AiServices itself, and this also led to a more cumbersome API. The idea here is that how the transactions among the different states are modeled should be totally external to the AiServices. This increased flexibility also allows to model the state machine with other external tools, like a rule engine in my case.
Note that this pull request (like the original one) still lacks the possibility to define a state machine for each user/conversation, but I believe this could be implemented with an id-based mechanism, very similar to the one already used for the chat memory. Moreover, since in my test example I'm using ollama that doesn't support tools, I didn't try to put also them under the control of the state machine, but once we agreed on the implementation this could also be added in a relatively easy way.
While working on this feature however I realized that probably also the concept of a proper state machine is quite over-engineered for the langchain4j needs, and what would be necessary is more simply a way to make system and user messages (and eventually tools) configurable in a programmatic way. I developed this idea in another pull request.