Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce latency for metric ingestion #3364

Open
jan-auer opened this issue Apr 2, 2024 · 0 comments
Open

Reduce latency for metric ingestion #3364

jan-auer opened this issue Apr 2, 2024 · 0 comments

Comments

@jan-auer
Copy link
Member

jan-auer commented Apr 2, 2024

The ingestion pipeline causes a delay of 1-2 minutes for metrics. Users expect metrics to show up earlier in the UI.

Especially if you want to have alerts or reactive functionality like canary deploys based on DDM, metrics need to be as high resolution and as low latency as possible. Since DDM has a resolution of 10s resulting from its bucketing window, users expect an ingestion delay roughly on the same level.

The delay results from bucketing and additional delays in both PoP and processing Relays. The theoretical minimal delay with current configuration is 70s in addition to any delays that the indexer and snuba consumers add.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Development

No branches or pull requests

1 participant