Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[DOC] Update docs to include local-blocks for metrics generator #3401

Closed
knylander-grafana opened this issue Feb 16, 2024 · 3 comments · Fixed by #3682
Closed

[DOC] Update docs to include local-blocks for metrics generator #3401

knylander-grafana opened this issue Feb 16, 2024 · 3 comments · Fixed by #3682
Assignees
@knylander-grafana
Labels
keepalive Label to exempt Issues / PRs from stale workflow type/docs Improvements or additions to documentation

Comments

@knylander-grafana
Copy link
Contributor

knylander-grafana commented Feb 16, 2024 •
edited

During a thread about Prometheus compatibility on Slack, several people noted issues in the docs and provided feedback:

Metrics-generator.storage.path and local-blocks

Slack thread

metrics_generator.storage.path is available in the config documentation: https://grafana.com/docs/tempo/latest/configuration/#metrics-generator
But for local-blocks there is no documentation. So all settings are missing.
metrics_generator.traces_storage.path is a required setting for local-blocks, but it not available in the documentation.
It is useful to add it to:
https://grafana.com/docs/tempo/latest/configuration/#metrics-generator
https://grafana.com/docs/tempo/next/api_docs/metrics-summary/ as here is some documentation provided for local-blocks, but only 1 part of the settings is mentioned.
https://grafana.com/docs/tempo/next/operations/prometheus_datasource/ is missing a reference on how to enable local-blocks, it only mentions "Once the local-blocks processor is enabled" . A link to documentation on how to enable it, would help a lot.

Prometheus data source

Resources
@knylander-grafana knylander-grafana added the type/docs Improvements or additions to documentation label Feb 16, 2024
@knylander-grafana
Copy link
Contributor Author

Additional feedback from Slack thread

From this comment:

At NEXT Version docs metrics-generator section, there is only this info.
The local blocks processor stores spans for a set period of time and enables more complex APIs to perform calculations on the data. The processor must be enabled for certain metrics APIs to function.
and metrics summary api section , It is described as if the metric summary api works well if just only this setting is enabled.

overrides:
  defaults:
    metrics_generator:
      processors: [..., 'local-blocks']

But when i test, metrics generator failed to start.
I confirmed in Github issue that I need to add the settings below to solve this problem.

metrics_generator:
  storage:
    path: /tmp/tempo/generator/wal
  traces_storage:
    path: /tmp/tempo/generator/traces

Thank you for reading it. that`s what i feel when reading docs.
And I have a few questions.
When I tested it, I think I can use the metrics summary api only for 1 hour data, right?
And I think the local-block processor is a stateful workload, so should I put a volume on the metrics-generator?

zleslie
2 days ago
@qqwweett
Thanks for the feedback. The metrics summary API only supports a 1 hour query, that's correct. Is the specific issue you mention that we don't have default value for metrics_generator.storage.path and metrics_generator.traces_storage.path? If so, would you be willing to file an issue on this?
The Prometheus compatibility that started this thread is a different API than the metrics summary API and supports full backend search. Though as Marty mentioned, we'll be looking to improve the performance as we move forward.

@knylander-grafana knylander-grafana self-assigned this Feb 16, 2024
@knylander-grafana knylander-grafana mentioned this issue Feb 21, 2024
Merged
3 tasks
@github-actions GitHub Actions
Copy link
Contributor

This issue has been automatically marked as stale because it has not had any activity in the past 60 days.
The next time this stale check runs, the stale label will be removed if there is new activity. The issue will be closed after 15 days if there is no new activity.
Please apply keepalive label to exempt this Issue.

@github-actions github-actions bot added the stale Used for stale issues / PRs label Apr 17, 2024
@knylander-grafana knylander-grafana added keepalive Label to exempt Issues / PRs from stale workflow and removed stale Used for stale issues / PRs labels Apr 30, 2024
@knylander-grafana
Copy link
Contributor Author

knylander-grafana commented May 16, 2024 •
edited

Content is added here: #3401

Published here: https://grafana.com/docs/tempo/latest/operations/traceql-metrics/#configure-the-local-blocks-processor

@knylander-grafana knylander-grafana mentioned this issue May 16, 2024
Merged
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@knylander-grafana knylander-grafana

Labels
keepalive Label to exempt Issues / PRs from stale workflow type/docs Improvements or additions to documentation
Projects
Tempo squad
Status: Done
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[DOC] Add link to local-blocks in metrics-generator knylander-grafana/tempo-doc-work
1 participant
@knylander-grafana