Connecting Elyra with Ray

[DhavalRepo18]

As a Data scientist, I would like to use Elyra with Ray (now also part of ODH) as the backend. Is there any plan to support Ray as a backend for the execution of the pipeline? I am happy to walk over the additional req.

[kevin-bates]

Hi @DhavalRepo18 - there are no immediate plans to support Ray, but I'd like to gather some more information about your use case.

Is the idea that you'd essentially be running in "local mode" (not KFP or Airflow) but that the cluster in which each notebook-based node runs is a Ray cluster? Do you need support for Python or R nodes and that they too run on Ray?

If the answer to the first question is "yes", and the second question is "no" then one way this may be achieved is to write a process proxy for Enterprise Gateway (or a gateway provisioner) that manages a kernel's lifecycle within a Ray cluster.

[DhavalRepo18]

@kevin-bates thanks for your response.

[lresende]

@DhavalRepo18 I would actually be interested in your requirements, what's the best way for us to connect? Would you also be willing to help with some integration here?

[lresende]

BTW, I believe we have some very minor integration as part of https://github.com/CODAIT/covid-notebooks/blob/master/README.md done by @frreiss

[DhavalRepo18]

@lresende, thanks for your pointer. Yes, I did look at that stuff when he was presenting. If you have any forum to present the idea, I can talk over it.

[DhavalRepo18]

@DhavalRepo18 I would actually be interested in your requirements, what's the best way for us to connect? Would you also be willing to help with some integration here?

Happy to help if needed.