Replies: 4 comments
-
So I know exactly what you're talking about. it's an orchestration / glue layer that's often defined as functions. It's also sometimes defined as a Flow or Workflow and often coincides with things like the Sagas patterns. I think AWS StepFunctions is the closest equivalent and they have a state language for this https://docs.aws.amazon.com/step-functions/latest/dg/concepts-amazon-states-language.html. Personally I think we're talking about API orchestration, I've thought about this in multiple forms:
We had someone define a "Flow" interface for Go Micro at one point but it was never merged https://github.com/unistack-org/micro/blob/v3/flow/flow.go#L127. I think the complexity of such a thing makes it hard to do upfront without offloading much of that complexity to the user. My guess is, the simplest model is user defined functions that give the developer freedom to code as they wish, in any language and combine APIs. Whatever we do, this sounds like something that should definitely be explored |
Beta Was this translation helpful? Give feedback.
-
It is worth the exploration I think, what you will end up with is a successful business model like the LEGO parts that can be combined together in many different ways. |
Beta Was this translation helpful? Give feedback.
-
I just noticed this thread. One thing I'm realising is there's probably a simpler, faster and more programmatic way to do this orchestration or even API layer creation. I think the Micro services aspect for this is useful where there's a large surface area to cover or complexity to manage but in a lot of cases where it's really just a simplified gateway, we need something else. I'm tempted to create an api package in this repo that enables some of what we're talking about. Just something tells me there's an opportunity to create a simplified gateway. import "github.com/m3o/m3o/api"
gw := api.New()
gw.Register(&api.Service{
Name: "opensea",
Handler: func(w http.ResponseWriter, r *http.Request) {
// handle req
}
})
gw.Run(ctx) Also the lego block analogy makes total sense. So if there's a way to chain, huge gains. Still so hard to do even with Micro. All the past efforts only made single service calls easy with service discovery, but it didn't take into account calling multiple services, chaining, transformations. I know things like GraphQL are supposed to help here but its just not programmatic. To do it with a library makes total sense. |
Beta Was this translation helpful? Give feedback.
-
To add to this. I felt massive ease of use with the micro cli from a consumption perspective. I could call anything from the CLI at rapid speed. I could use unix pipes to transform some of that but I could not pipe one service output into the input of another. I also could not programmatically save any of that for future use. I really wish I had fast and easy ways to transform, create new apis, etc, etc. It really is a new layer. I don't know whether part of that is packaging as wasm binaries or using similar to https://github.com/fermyon/spin |
Beta Was this translation helpful? Give feedback.
-
The way
It is time to think about a new layer on top of currently available clients that we have in order to simplify things like M3O service itself and the frontend/backend user experience.
Frontend/Backend developers need something to glue their own pieces together like for instance the user registration process for an app using a smartphone, obviously you need to make multiple calls to M3O services in order to achieve that and complete the cycle, but what if we provide a new mechanism that will facilitate the whole process, what if we can reduce the number of pieces all together.
The How
We can introduce a new layer on top of currently available clients we have in a way that will allow frontend/backend developers to create new custom services from currently available M3O services.
We call it M3O agent/sidcar for now and it has these elements/properties:
Runs using ONE special token.
An interface called Orchestrator.
A SeqOrch structure that satisfies an Orchestrator interface: a sequence orchestrator where you can add multiple service wrappers and these wrappers will be called in sequence.
A ParOrch structure that satisfies an Orchestrator interface: a parallel orchestrator where you can add multiple service wrappers and these wrappers will be called in parallel.
A CustomService structure: a custom service structure that has a method called Execute which accepts multiple Orchestrators or a DSL file.
A service wrapper: which wraps an M3O service client, it will obtain/check the underlying service token and it has a function that will be called before the next wrapper in the queue.
This will allow us/frontend/backend to create new custom services from basic blocks that M3O provides and it will keep M3O basic blocks simple as well.
Beta Was this translation helpful? Give feedback.
All reactions