-
Notifications
You must be signed in to change notification settings - Fork 472
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
The individual components should be deployable without setting the executor first #583
Comments
@karakanb The values validations are intended to ensure people don't deploy non-sensible combinations of values in the chart (as there are MANY options). Is there a reason why you can't set your I believe you can change your DAG tasks to use the
|
I can do that, but that means that I have to deploy all of the changes at once, including potentially blocking the dag runs in case something is wrong, e.g. a task being forgotten to moved to a default queue, or a task setting an explicit queue somewhere; therefore, it seems to be the safer option to be able to incrementally deploy things and ensure they work step by step without breaking the current flow. Would you be interested in a solution like |
Checks
User-Community Airflow Helm Chart
.Motivation
I am trying to transition from KubernetesExecutor to CeleryKubernetesExecutor gradually. My goal is to do this transition as safe as possible without any disruptions, and do every change incrementally.
What I want to do is:
kubernetes
queue, so that they'll behave the same way.CeleryKubernetesExecutor
I would like to be able to do this, because at each step I want to make sure all is good before I transition the pipelines, and I have a mixed set of workloads that I would like to keep on Kubernetes, while ensuring that after the rollout everything will keep running the same way.
As of now, I am not able to do this gradual switch because of the simple check on the helm chart.
Implementation
Remove the check for disabling workers / redis / flower.
Are you willing & able to help?
The text was updated successfully, but these errors were encountered: