GH-2381 : Support kafka parallel-consumer #3161
Open
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Introduction
Hi, Spring Team.
This is chickenchickenlove, huge fan of you guys.
I saw this issue (Support kafka parallel-consumer), and this comment as well!
So, i'd like to implement that integrate
Parallel Consumer
tospring-kafka
This is skeleton code.
Please take a look when you have some free time. πββοΈ
Thank you for your time!
Background
Parallel consumer
will be more helpful than kafka consumer, because parallel consumer have good parallel processing (If you want to know more, See this).spring-kafka
as well.How To use
@EnableParallelConsumer
to your Spring application. then, some spring bean related withparallel consumer
, will be registered toApplicationContext
ParallelConsumerCallback
and register their concrete class as spring bean by annotating@Component
or@Configuration
.ParallelConsumerFactory
will startParallelConsumer
andParallelConsumer
process records withParallelConsumerCallback
concrete class.ETC
@KafkaListener
?ParallelConsumer
can allow only one callback. thus, it is not suitable forParallelConsumer
.ParallelConsumerImportSelector
?org.springframework.boot.autoconfigure.kafka.KafkaAutoConfiguration.java
class support to register spring bean,ParallelConsumerImportSelector
can be removed.Update from Apr 27.
parallelConsumer
.Thank you for your time, Again.