CompleteRequestSettings is insufficiently abstract #2626
Replies: 1 comment
-
Hi @douglasware, did you see about that continuation of the OobaboogaConnector, and the additional wrapped updates to CompleteRequestSettings being applied in the related MultiConnector. |
Beta Was this translation helpful? Give feedback.
-
When I started using other completion APIs, e.g. Oobabooga, HuggingFace, the design around Microsoft.SemanticKernel.AI.TextCompletion.CompleteRequestSettings gets inconvenient quickly because behind those APIs the parameters vary, and the SK implementation is specifically based on openai. Are there plans to fix this? In my fork I extended the class but that is a fragile approach, and it requires changes to the core kernel abstractions each time.
I haven't thought about it deeply, but something dynamic against the schema from the openapi description?
Seems like this one design element is a big blocker to the vision of a pluggable system.
Also, I think it would be nice to be able to set most of the settings in the constructor of the connector and override (or add additional) settings such as various logit warpers on individual completion requests.
Adding these other model sources into the mix challenges another aspect of the current design which is the tight coupling of plugin text and the resulting prompts to a specific API provider such as OpenAI instead of to a specific model. Prior to the introduction of Connectors.AI.Oobabooga, I wrote my own simple connector https://github.com/douglasware/ElumenotionSite/tree/master/Projects/OobaboogaAPIHelper and in it I separated the prompt text and the prompt templating. The separation of concerns makes it possible to use the turn text against different models with different turn templates. I'm not saying SK should take my implementation, just that it illustrates the challenge. As it stands, it is not possible on a practical level to use Connectors.AI.Oobabooga against arbitrary models loaded in Oobabooga Web UI without cracking it open and adding stuff.
Beta Was this translation helpful? Give feedback.
All reactions