Discrepancy between Thanos recording rule and underlying query (#2319) · Issues · GitLab.com / GitLab Infrastructure Team / Observability / Observability Issue Tracker · GitLab (original) (raw)

Skip to content

GitLab Next

• • Why GitLab
  • Pricing
  • Contact Sales
  • Explore
• Why GitLab
• Pricing
• Contact Sales
• Explore
• Sign in
• Get free trial

Discrepancy between Thanos recording rule and underlying query

Originally from #2264 (comment 1369564914). @msmiley noticed apdex ratios being more than 1 in https://gitlab.com/gitlab-com/gl-infra/scalability/uploads/17eb3220684035585ecf3eaab7c4450c/Screenshot_from_2023-04-26_16-56-56.png.

Digging deeper, he found that the denominator of the apdex calculations gitlab_component_apdex:weight:score_5m, differed from the underlying query for redis-cluster-ratelimiting service.

Thanos source

/cc @reprazent @msmiley

Some similar issues on thanos github

• https://github.com/thanos-io/thanos/issues/3859
• https://github.com/thanos-io/thanos/issues/4698

Edited Apr 27, 2023 by Sylvester Chin