-
|
Hi, I'm currently evaluating flyte for managing our workflows and tasks which are currently based on docker container. It's not clear for me if flyte (service) can handle a workflow as a single container. Thanks in advance |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 4 replies
-
|
@ncouronne IIUC, you would like in some cases that the entire workflow simply runs in one container. Currently, Flyte does not support automatically fusing multiple tasks into a single container. But you can achieve this using Subworkflows and Local workflow execution. At the moment, from Flyte's point of view this will look like a single task - wherein you can execute the subworkflow in Local Mode and then return. So in case of retries the entire "task + Subworkflow" will be retried. To reduce the penalty of a re-run we are working on Intratask checkpointing and eventually we will be supporting Automatic Task fusion or Coalesce We would love to work with you as a community and try and deliver these features. As a community we believe in building features for users and usecases, that way we can get iterative and quick feedback. Please join our slack channel and we would love to work with you |
Beta Was this translation helpful? Give feedback.
-
|
Also to clarify to execute a workflow in local mode you just have to invoke it - within a task. |
Beta Was this translation helpful? Give feedback.
-
|
@kumare3 when you speak about local, you mean with the sandbox? i'd like to join the slack channel, but it seems that someone shall invit me :( |
Beta Was this translation helpful? Give feedback.
-
|
Ohh that's not the case, which link did you use? In any case you can use this link |
Beta Was this translation helpful? Give feedback.
-
|
Also by local I mean, flytekit execution. Join the slack and we can try things |
Beta Was this translation helpful? Give feedback.
@ncouronne IIUC, you would like in some cases that the entire workflow simply runs in one container. Currently, Flyte does not support automatically fusing multiple tasks into a single container. But you can achieve this using Subworkflows and Local workflow execution. At the moment, from Flyte's point of view this will look like a single task - wherein you can execute the subworkflow in Local Mode and then return. So in case of retries the entire "task + Subworkflow" will be retried.
To reduce the penalty of a re-run we are working on Intratask checkpointing and eventually we will be supporting Automatic Task fusion or Coalesce
We would love to work with you as a community and try and de…