Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Some questions about S3 #497

Open
Duansg opened this issue May 8, 2023 · 2 comments
Open

Some questions about S3 #497

Duansg opened this issue May 8, 2023 · 2 comments
Labels
question Further information is requested

Comments

@Duansg
Copy link

Duansg commented May 8, 2023

I saw that rocketmq connect was designed as a tool for streaming data across heterogeneous systems. But I have a few questions, as follows:
1、Now, the connectors support only partial heterogeneous systems of data processing? Because I have looked at some of the code related to sink task in the master branch, such as (sftp, cassandra)sink task, their source/sink task is processed in a special data way. It may not support things like redis source -> sftp sink this way.
2、SSo, for S3, need to support S3 source -> S3 sink this way? Or do you need to support all heterogeneous systems?
3、If need to support other heterogeneous systems, like S3 sink for batch message processing such as Mysql source, it needs to generate table files?

@odbozhou
Copy link
Contributor

odbozhou commented May 8, 2023

You can look at the jdbc connector implementation, which is relatively complete. It is not difficult to achieve data transmission between heterogeneous systems, mainly through the support of schema and converter.

@odbozhou odbozhou added the question Further information is requested label May 8, 2023
@Duansg
Copy link
Author

Duansg commented May 8, 2023

@odbozhou ok,thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants