Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

changed Kernel info breaks querie(s) #397

Open
roock opened this issue Jan 9, 2024 · 1 comment
Open

changed Kernel info breaks querie(s) #397

roock opened this issue Jan 9, 2024 · 1 comment

Comments

@roock
Copy link
Contributor

roock commented Jan 9, 2024

I observed this behavior with the "Host disk will fill in 24 hours", but it might also affect other alerts using node_uname_info.

This is happening in the following conditions:

  • the servers is rebooted with a new(er) kernel and the version/release information changes
  • the servers has multiple monitored partitions e.g. rootfs and /srv
execution: found duplicate series for the match group {instance="monitor.localdomain:9100"} on the right hand-side of the
operation:
[{__name__="node_uname_info", domainname="(none)", group="infra", instance="monitor.localdomain:9100",
job="node", machine="x86_64", nodename="monitor", release="5.10.0-27-amd64", sysname="Linux",
version="#1 SMP Debian 5.10.205-2 (2023-12-31)"},
{__name__="node_uname_info", domainname="(none)", group="infra", instance="monitor.localdomain:9100",
job="node", machine="x86_64", nodename="monitor", release="5.10.0-26-amd64", sysname="Linux",
version="#1 SMP Debian 5.10.197-1 (2023-09-29)"}];
many-to-many matching not allowed: matching labels must be unique on one side
@guruevi
Copy link

guruevi commented Feb 25, 2024

I believe the reason is this part of the query:
on(instance) group_left (nodename) node_uname_info{nodename=~".+"}

It is probably unnecessary and should be handled by relabeling in Prometheus or using a regexp.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants