Support for Cyclic Dependencies in Kedro Pipelines for Reinforcement Learning Scenarios

[Sino-Huang]

Description

I'm currently facing challenges with the Kedro pipeline structure, specifically its limitation to Directed Acyclic Graphs (DAGs). In reinforcement learning applications, the ability to create cyclic loops within the pipeline is crucial. For instance, a learning policy generates data that is then used to further train and refine the same policy. The current DAG structure does not support these types of cyclic dependencies, which is limiting for projects that involve iterative data generation and processing loops.
Context

The addition of support for cyclic dependencies is important because it would allow for more flexible pipeline configurations, especially beneficial in the context of AI and machine learning projects where iterative feedback loops are common. This feature would not only benefit my projects but also broaden Kedro's applicability in advanced machine learning scenarios, promoting its adoption and enhancing its utility.
Possible Implementation

One way to implement this could be by allowing users to define nodes or sub-pipelines that can conditionally loop back to earlier stages based on runtime data or conditions. Or let's have a counter to count the loop.

[astrojuanlu]

Hi @Sino-Huang , thanks for opening this! I converted it to a discussion.

Typically, what other frameworks exist for reinforcement learning? How do you define your convergence criteria so that the pipeline eventually stops?

[Sino-Huang]

Good day, I see no framework support this cyclic dependencies feature. I looked at Zenml, but they seem not having this feature either.

So I will have a Budget-Based Exit Strategy
e.g. Fixed Time Budget: Set a fixed time limit
for your process. If the iteration exceeds this time, terminate it.

Or Budget Limit Iteration Number: Define a maximum number of iterations (budget).Once this limit is reached, exit the loop