Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modify k8s API caches and watchers to only store relevant projections of pod #2642

Open
AjayTripathy opened this issue Mar 15, 2024 · 0 comments
Labels
E3 Estimated level of Effort (1 is easiest, 4 is hardest) kubecost Relevant to Kubecost's downstream project opencost OpenCost issues vs. external/downstream P2 Estimated Priority (P0 is highest, P4 is lowest)

Comments

@AjayTripathy
Copy link
Contributor

AjayTripathy commented Mar 15, 2024

Is your feature request related to a problem? Please describe.
#2637

Describe the solution you'd like
We don't use all the data out of the k8s API for each pod. Cutting out unnecessary fields can dramatically reduce the memory profile of opencost. However, I'm not sure if that's supported out of the box in the k8s caching watchers; more investigation would be needed.

Estimated impact:

Opencost seems to keep almost every Kubernetes object fully in memory, but only a few fields are used. The watcher and cache took up 56% of the memory, while the metrics emitter used only 4%.

I think we could probably halve the cache memory... so 25% improvement

@mattray mattray added opencost OpenCost issues vs. external/downstream P2 Estimated Priority (P0 is highest, P4 is lowest) kubecost Relevant to Kubecost's downstream project E3 Estimated level of Effort (1 is easiest, 4 is hardest) and removed needs-triage needs-follow-up labels Apr 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
E3 Estimated level of Effort (1 is easiest, 4 is hardest) kubecost Relevant to Kubecost's downstream project opencost OpenCost issues vs. external/downstream P2 Estimated Priority (P0 is highest, P4 is lowest)
Projects
None yet
Development

No branches or pull requests

2 participants