New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
api/v1/metadata and api/v1/label/__name__/values returning different values #14013
Comments
Yes. Metadata is associated with a "metric family" (just the name of the metric) while It's reasonable as a feature request: "if all series for a family are dropped, ignore the metadata", but not easy to implement. |
What did you do?
I have a prometheus target with hundreds of thousands of metrics (
MY_TARGET
in the config below). I do not have control over the metrics being exposed and I only care about a handful of metrics that contain labelMY_LABEL
I created the
metric_relabel_configs
below to drop all metrics that do not contain a specific label (MY_LABEL
).The config above works and the metrics I don't care about are not available when I call
api/v1/query
.When I open https://PROMETHEUS_URL/graph with
Enable autocomplete
checked, it makes 2 calls to prometheus APIhttps://PROMETHEUS_URL/api/v1/label/__name__/values
https://PROMETHEUS_URL/api/v1/metadata
The first call correctly returns a small list containing the names of only the metrics I care about.
The second (v1/metadata), returns all metrics, including those that have been dropped by the config above.
What did you expect to see?
I would have expected
v1/metadata
to return the same list asapi/v1/label/__name__/values
but instead if returns hundreds of thousands more. All of the ones that have been dropped by the config above.Is it possible that
api/v1/metadata
is being populated BEFORE the metrics are being dropped by the config above?What did you see instead? Under which circumstances?
.
System information
Linux 4.18.0-372.78.1.el8_6.x86_64 x86_64
Prometheus version
Prometheus configuration file
No response
Alertmanager version
No response
Alertmanager configuration file
No response
Logs
No response
The text was updated successfully, but these errors were encountered: