-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[DS][x/n] Allow SchedulingConditions to target specific dependency keys #21854
Conversation
Warning This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
This stack of pull requests is managed by Graphite. Learn more about stacking. Join @OwenKephart and the rest of your teammates on Graphite |
if not self.dep_selection.is_all: | ||
description += f" within selection: {self.dep_selection.description}" | ||
return description |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: could be one-liner
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe missing something dumb, but I'm not seeing a clean way to make this a oneliner, e.g.:
return "All deps" + (f"within _selection: {self.dep_selection}" if self.dep_selection else "")
would work, but it feels a fair bit harder to read to me. Open to suggestions though
python_modules/dagster/dagster/_core/definitions/declarative_scheduling/scheduling_condition.py
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We need to figure out a way to make this compose better IMO
b75b319
to
0ec11e5
Compare
f66fb78
to
c100c16
Compare
0ec11e5
to
6b48cc1
Compare
c100c16
to
ae60aef
Compare
6b48cc1
to
d06ca84
Compare
d06ca84
to
1bc7987
Compare
@schrockn Updated this to just work off of regular-old |
ae60aef
to
c3080f8
Compare
c3080f8
to
8f3fbc9
Compare
1bc7987
to
abdf548
Compare
06e224c
to
83934b5
Compare
83934b5
to
1b5c3d4
Compare
Summary & Motivation
This is a long-standing request. It's often useful to be able to detect the status of some specific set of dependency keys, rather than needing to treat all dependencies identically.
This creates a single structure for defining and resolving these sorts of collections, but does not export it as a top-level user-facing concept, as that feels a bit like jumping the gun.
This allows users to explicitly select any keys they want to select for, as well as explicitly ignore some specific keys, as either workflow is potentially useful. An explicit choice was made to not error when an "invalid" selection (i.e. one that references keys that are not dependencies) is made, as this would be pretty annoying (imo).
How I Tested These Changes