-
Notifications
You must be signed in to change notification settings - Fork 359
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
AWS S3 sink connect surrounds messages with quotes #874
Comments
We've implemented a solution here: INSERT INTO target_topic SELECT * FROM mkc-tutorial-destination-bucket:tutorial STOREAS `JSON` WITH_FLUSH_INTERVAL = 60 WITH_FLUSH_COUNT = 100 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Issue Guidelines
How can I configure the S3 Sink to write messages to S3 objects without surrounding quotes? I would expect the
source_topic
andtarget_topic
to be identical when using the sink and source connector.following this aws blog post, more or less.
What version of the Stream Reactor are you reporting this issue for?
I built a fat jar from master branch. The 3.0.1 release won't work because my bucket is in us-west-2 and I found that only the latest build supports the
connect.s3.aws.client=AWS
property.Are you running the correct version of Kafka/Confluent for the Stream reactor release?
yes
Have you read the docs?
yes
What is the expected behaviour?
The S3 source connector loads objects messages from S3 to Kafka without surrounding quotes.
What was observed?
What is your Connect cluster configuration (s3-sink-connector.properties)?
This is what the bucket objects contain:
What is your connector properties configuration (s3-source-connector.properties)?
The text was updated successfully, but these errors were encountered: