You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
find_pipelines is added since 0.18.3 as a pipeline autodiscovery feature. While this is useful, it is sometimes hard to debug because it raises warning but not error so the program will keep running. Cc @deepyaman
Context
For library developers, they may need strict control and want the pipeline fails explicitly. This features was designed for development, so any half-developed pipeline won't prevent other pipeline from running (incomplete pipeline, not installed library etc)
Some quotes from users:
User 1 (library developer)
We recently have users reporting issues when running tensorflow on M1. The error was hidden in some kedro pipelines and it took a while to dig out the root cause from the user. In general, it can be difficult to predict when and where we will encounter these issue. Therefore, we are planning to build some CI workflows for checking our packages and pipelines for M1 machines in general.
User 2 (general user)
I'm noticing that warnings like SyntaxErrors and type errors are considered "warnings" by Kedro, which continues to try to run the pipeline. Is there a setting to escalate these to Errors so they can abort the pipeline run? 15 days ago
Is there a way to mark a pipeline as a work in progress and then tell find_pipelines to error on warnings? Or is the answer to just not use pipeline autodiscovery
Description
find_pipelines
is added since 0.18.3 as a pipeline autodiscovery feature. While this is useful, it is sometimes hard to debug because it raises warning but not error so the program will keep running. Cc @deepyamanContext
For library developers, they may need strict control and want the pipeline fails explicitly. This features was designed for development, so any half-developed pipeline won't prevent other pipeline from running (incomplete pipeline, not installed library etc)
Some quotes from users:
User 1 (library developer)
User 2 (general user)
Possible Implementation
find_pipelines(raise_errors=False)
- default unchanged, optionally fail eagerly.Possible Alternatives
The text was updated successfully, but these errors were encountered: