Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(relay): Move histogram outliers to global config #71004

Merged
merged 12 commits into from
May 22, 2024

Conversation

jjbayer
Copy link
Member

@jjbayer jjbayer commented May 16, 2024

Histogram outliers are a static piece of metrics extraction config that can be moved to global config. This will reduce network traffic, project config sizes in redis and first and foremost relay in-memory caches.

This PR also fixes a long-standing bug in the outlier config.

ref: https://github.com/getsentry/team-ingest/issues/331

@github-actions github-actions bot added the Scope: Backend Automatically applied to PRs that change backend components label May 16, 2024
@@ -1427,37 +1433,48 @@ def _produce_histogram_outliers(query_results: Any) -> Sequence[MetricConditiona
# See also https://en.wikipedia.org/wiki/Outlier#Tukey's_fences
{
"op": "gte",
"name": "event.duration",
"name": _HISTOGRAM_OUTLIERS_SOURCE_FIELDS[metric],
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This fixes a bug that went unnoticed for a long time: p25 and p75 are either duration, LCP, or FCP, but the value we compare them to is always duration.

@@ -1013,7 +1013,7 @@ def _filter_option_to_config_setting(flt: _FilterSpec, setting: str) -> Mapping[
#: When you increment this version, outdated Relays will stop extracting
#: transaction metrics.
#: See https://github.com/getsentry/relay/blob/6181c6e80b9485ed394c40bc860586ae934704e2/relay-dynamic-config/src/metrics.rs#L85
TRANSACTION_METRICS_EXTRACTION_VERSION = 5
TRANSACTION_METRICS_EXTRACTION_VERSION = 6
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Version bump needed because old Relays do not apply globally defined tags to legacy metrics (see getsentry/relay#3615).

pass

consumer.close()
assert histogram_outlier_tags == {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This assertion will fail until getsentry/relay#3615 is merged (requires up to date relay docker image).

@jjbayer jjbayer marked this pull request as ready for review May 17, 2024 09:28
@jjbayer jjbayer requested review from a team as code owners May 17, 2024 09:28
jjbayer added a commit to getsentry/relay that referenced this pull request May 21, 2024
Prerequisite to getsentry/sentry#71004: Global
tag mappings were not yet being applied to transaction metrics.

Beside the bug fix, this PR also bumps the metrics extraction
version(s), and attempts to simplify the extraction function for
transactions.
Copy link

codecov bot commented May 22, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 77.86%. Comparing base (f535fed) to head (f8a70e8).

Additional details and impacted files
@@           Coverage Diff           @@
##           master   #71004   +/-   ##
=======================================
  Coverage   77.85%   77.86%           
=======================================
  Files        6533     6532    -1     
  Lines      291110   291110           
  Branches    50377    50372    -5     
=======================================
+ Hits       226657   226660    +3     
+ Misses      58218    58211    -7     
- Partials     6235     6239    +4     
Files Coverage Δ
src/sentry/relay/config/__init__.py 91.94% <100.00%> (ø)
src/sentry/relay/config/metric_extraction.py 90.48% <100.00%> (+0.20%) ⬆️
src/sentry/relay/globalconfig.py 96.55% <100.00%> (+0.25%) ⬆️
src/sentry/snuba/metrics/extraction.py 90.84% <100.00%> (+0.03%) ⬆️

... and 16 files with indirect coverage changes

@jjbayer jjbayer removed the Do Not Merge Don't merge label May 22, 2024
@jjbayer jjbayer merged commit 6346c17 into master May 22, 2024
51 checks passed
@jjbayer jjbayer deleted the feat/global-histogram-outliers branch May 22, 2024 11:07
@github-actions github-actions bot locked and limited conversation to collaborators Jun 6, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Scope: Backend Automatically applied to PRs that change backend components
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants