-
Notifications
You must be signed in to change notification settings - Fork 135
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Recommendations missing for some sidecars #276
Comments
Hi, do you see anything when running krr w/ Any pattern to the sidecars? E.g. init containers vs regular contianers. |
Hi @aantn, They are regular containers in the sense that they are mentioned under |
Got it. So they are present in |
Yes, correct. |
Does it work on #266 if you choose the Prometheus loader? That will load info about workloads from Prometheus where the data on actual containers in the pod should be present. |
Of course, we would prefer to have recommendations being based on the longer term Thanos data, but it looks like we can give the Thanos URL als Prometheus URL, is that correct? |
Yes, exactly! Can you try and confirm that it works? |
My first few tests gave identical recommendations using both Thanos and Prometheus, but I found another cluster where Prometheus didn't contain enough data to give recommendations but Thanos did :) So yes, it works. Looking forward to seeing this merged and available via brew. |
Wonderful, thank you. To help get this merged, are you able to run a 3 way test:
We're doing testing to confirm that there aren't unexpected regressions in the new branch which includes some very large changes to the codebase. So getting another confirmation on this from real world data would be extremely valuable. |
When performing the test, I actually observed that the current data without prometheus discovery is wrong for the pods that contain the multiple containers as discussed in this issue. This goes for both the master and the new branch: According to the report, the both current request and memory limit for the main container are set at 1024Mi, while actually the request is 256 and the limit 512 (missing sidecar container 128/256). This error happens for multiple (but strangely enough not all) containers with this Dapr sidecar pattern. This makes this issue slightly more pressing than that there are just some containers missing. Next to this, the 'discovery' option give me a lot of recommendations for resources that were not present in both non-discovery tests, but that is probably the point of this whole exercise :) Apart from that, the recommendations given by all three are equal. |
Thanks. Can you share the lines from the reports which are wrong? Sounds
like the prom discovery is working but we should also get that bug fixed
for the api server method. (We likely will continue to exclude injected
containers from the report, but the data on other containers should be
correct.)
…On Wed, May 15, 2024, 19:19 Pieter ***@***.***> wrote:
When performing the test, I actually observed that the current data
without prometheus discovery is wrong for the pods that contain the
multiple containers as discussed in this issue. This goes for both the
master and the new branch:
According to the report, the both current request and memory limit for the
main container are set at 1024Mi, while actually the request is 256 and the
limit 512 (missing sidecar container 128/256). This error happens for
multiple (but strangely enough not all) containers with this Dapr sidecar
pattern. This makes this issue slightly more pressing than that there are
just some containers missing.
Next to this, the 'discovery' option give me a lot of recommendations for
resources that were not present in both non-discovery tests, but that is
probably the point of this whole exercise :)
Apart from that, the recommendations given by all three are equal.
—
Reply to this email directly, view it on GitHub
<#276 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AADYUB3QSYQ7GJNWLFRKJU3ZCODJNAVCNFSM6AAAAABHMWYZ4OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMJSHE3DMMZZHE>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Describe the bug
We do not get a recommendation for some containers if there are multiple of them in a pod, e.g. the Dapr sidecar. For other pods with multiple containers, e.g. the prometheus pod with the
thanos-sidecar
andconfig-reloader
sidecars, we do get recommendations. The missing sidecars are not mentioned in any logs.Expected behavior
Recommendations for all containers in a pod or at least a mention the there is no / not enough data to make a recommendation for that specific container.
The text was updated successfully, but these errors were encountered: