Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mediums are a mess #30

Open
asubb opened this issue Dec 27, 2019 · 0 comments
Open

Mediums are a mess #30

asubb opened this issue Dec 27, 2019 · 0 comments

Comments

@asubb
Copy link
Member

asubb commented Dec 27, 2019

There is a package inside exe project io.wavebeans.execution.medium which covers serialization issues for execution. It is complete mess by itself with self-written serialization.

The point of the medium is to provide functionality for effective transferring data for pods.

Current limitations:

  1. (for framework developer) It is very hard to add a new medium type onto the list, hence it is hard to add new types into the lib and have support of it during execution.
  2. (for framework user) It is impossible to add custom type if any, however lib functionality allows to do this.
  3. Despite the fact that current implementation is kinda fast (it's dumb though), it's definitely way faster than built-in java serialization, it is completely unsupportable.

What needs to be added:

  1. Transparent way for framework developers to extend the medium list without pain. Perhaps using 3rd party libraries. What should take into consideration: current implementation heavily reuses primitive arrays, otherwise that's a lot of pressure on GC (seems so).
  2. A way for framework users to add their own types and provide medium for this if that's required. Or better provide some generic medium by default.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant