Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make Kubernetes default resources configurable #1800

Merged
merged 5 commits into from May 6, 2024

Conversation

martinhausio
Copy link
Contributor

@martinhausio martinhausio commented Apr 11, 2024

This fixes #1769.

A quick addition that looks if the METAFLOW_KUBERNETES_CPU, METAFLOW_KUBERNETES_MEMORY, METAFLOW_KUBERNETES_DISK environment variable is set and if so parses the values. If a value is detected as being an attribute, then assign it, otherwise it will just use the default values as defined.

@martinhausio
Copy link
Contributor Author

martinhausio commented Apr 11, 2024

I have tested locally by doing pip install ., then instantiating the same workflow with and without the environment variable and this behaves as intended.

I tested setting cpu, memory and disk separately, and together successfully.

@saikonen saikonen self-requested a review April 30, 2024 16:15
@martinhausio
Copy link
Contributor Author

@saikonen , changes ready for your review!

@martinhausio martinhausio requested a review from saikonen May 2, 2024 16:07
metaflow/plugins/kubernetes/kubernetes_decorator.py Outdated Show resolved Hide resolved
metaflow/metaflow_config.py Outdated Show resolved Hide resolved
@martinhausio martinhausio requested a review from saikonen May 4, 2024 04:09
@saikonen saikonen merged commit ec1c45d into Netflix:master May 6, 2024
26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Make default kubernetes resources configurable
2 participants