Provide Message Based Cloud Event Emitter (maybe Kafka)

[salaboy]

The Cloud Event Router should provide an extension to be able to receive Cloud Events using different transports. Kafka might be a first good candidate.

[tomdavidson]

Apache Pulsar and GCP PubSub would be more interesting to my case but Im sure its a minority position compared to Kafka.

[panwarab]

I would like to take this up

[salaboy]

@tomdavidson @masterskipper sounds good to me.. I am wondering how the integration will work out.. as we should be able to support all of them if we have a plan for how the extension will work. I am happy if you guys want to work together on that.

[salaboy]

@masterskipper I would recommend listing the steps that you are planning to do before spending time in coding or sending PRs, so we can review it together.

[panwarab]

@salaboy From the description, the problem statement is not quite clear to me. Looks like event routing needs to be made flexible ( not sure what are high level requirements ).

[panwarab]

can you confirm the requirement?

• Event Router should listen as a consumer to a message queue and listen for the events.
• There can be different producers but we need to have some particular contract between the two.

[salaboy]

@masterskipper let's define the requirements here for clarity:

• The Event Router exposes an HTTP endpoint for consuming Cloud Events
• The Event Router emits Cloud Events via HTTP

The scope for this issue is to provide a Cloud Event Message-Based emitter that can send Cloud Events via Kafka, using probably Spring Cloud Streams to achieve that. This needs to work with the Knative Broker as the HTTP version is doing.

We cannot add Kafka dependencies straight into this project, which means that we will need to create a separate project (maven module) for the integration that can be added optionally to the cloud event router fat jar.

My suggestion will be to start by adding those dependencies to this project and then we split it up when it is working.

Hopefully, that makes sense, if not please feel free to add questions.