Skip to content

metrics: give each backoff config its own histogram label - #2171

Open
hsdfat wants to merge 1 commit into
tikv:masterfrom
hsdfat:metrics-backoff-labels-1802
Open

hsdfat wants to merge 1 commit into
tikv:masterfrom
hsdfat:metrics-backoff-labels-1802

Conversation

@hsdfat

@hsdfat hsdfat commented Nov 12, 2026 •

Copy link
Copy Markdown

close #1802

Four backoff configs (BoTxnNotFound, BoMaxTsNotSynced, BoCommitTSLag, BoMaxRegionNotInitialized) shared metrics.BackoffHistogramEmpty, an observer of the backoff histogram with an empty type label. Grafana renders that series as "Value", so these backoffs could not be told apart, or recognised at all.

Each config now has its own observer, labelled with the name the config already has: txnNotFound, maxTsNotSynced, commitTSLag, regionNotInitialized. This is the shape of #1862, which was approved once and then closed by its author.

Compatibility: metrics.BackoffHistogramEmpty is removed. Nothing else in this module referenced it. If a downstream project uses it, I can keep it as a deprecated alias instead; say the word. Dashboards that select type="" need the new label values.

Test: TestBackoffConfigsHaveNamedMetric checks that a backoff of each of the four configs is counted under its own name and that no series with an empty type label exists. It fails on master and passes with this change. go test --tags=intest -race ./config/retry/ and golangci-lint run ./config/... ./metrics/... are clean.

AI assistance: I used an AI assistant while writing this change and reviewed and tested it myself.

Summary by CodeRabbit

  • Bug Fixes
    • Backoff metrics for transaction-not-found, timestamp synchronization, commit timestamp lag, and region initialization now appear under distinct names instead of an empty label.
    • This makes these backoff measurements easier to identify and distinguish when monitoring system behavior.

BoTxnNotFound, BoMaxTsNotSynced, BoCommitTSLag and
BoMaxRegionNotInitialized shared one observer with an empty type label,
which Grafana shows as a series called "Value". Each now reports under
its config name.

BackoffHistogramEmpty is removed; nothing else in this module used it.

close tikv#1802

Signed-off-by: phatlc <phatle.hsd@gmail.com>
@ti-chi-bot ti-chi-bot Bot added the dco-signoff: yes Indicates the PR's author has signed the dco. label Nov 12, 2026
@ti-chi-bot

ti-chi-bot Bot commented Nov 12, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign zhouqiang-cl for approval. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot Bot added contribution This PR is from a community contributor. needs-ok-to-test Indicates a PR created by contributors and need ORG member send '/ok-to-test' to start testing. labels Nov 12, 2026
@ti-chi-bot

ti-chi-bot Bot commented Nov 12, 2026

Copy link
Copy Markdown

Hi @hsdfat. Thanks for your PR.

I'm waiting for a tikv member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work.

Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@ti-chi-bot ti-chi-bot Bot added the first-time-contributor Indicates that the PR was contributed by an external member and is a first-time contributor. label Nov 12, 2026
@ti-chi-bot

ti-chi-bot Bot commented Nov 12, 2026

Copy link
Copy Markdown

Welcome @hsdfat!

It looks like this is your first PR to tikv/client-go 🎉.

I'm the bot to help you request reviewers, add labels and more, See available commands.

We want to make sure your contribution gets all the attention it needs!



Thank you, and welcome to tikv/client-go. 😃

@coderabbitai

coderabbitai Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

Four backoff configurations now record observations under dedicated histogram labels instead of an empty label. New tests check that each named series receives a sample and that no empty-type series exists.

Changes: Backoff Metric Labels

Layer / File(s) Summary
Define named backoff shortcuts
metrics/shortcuts.go
Four observer shortcuts replace BackoffHistogramEmpty and bind to the matching labels on TiKVBackoffHistogram.
Use and verify named labels
config/retry/config.go, config/retry/config_test.go
Four backoff configurations use the new shortcuts. The test checks that each named series gains one sample and that no empty-type series exists.

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix · Severity of issue fixed: Low

Suggested reviewers: disksing, lcwangchao

Merge Risk: Minimal · up to f251e

This change gives four backoff metrics distinct histogram labels instead of an empty one. Dashboards filtering on the empty label value will need updating. No merge-blocking risk was found.

Pre-merge checks: Passed 4

Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title "metrics: give each backoff config its own histogram label" matches the main change. The PR replaces the shared metrics.BackoffHistogramEmpty observer with four distinct observers. Each one …
Linked Issues check Passed Issue #1802 asks to improve the Grafana label of the backoff histogram series shown as "Value" (the empty type label), possibly with reasonable names. The diff from merge base to head meets this. In…
Out of Scope Changes check Passed All changed hunks in the inspected diffs serve the #1802 objective. metrics/shortcuts.go removes BackoffHistogramEmpty and adds four named observers. config/retry/config.go rebinds only the four…
Full details: Title check

Explanation

The title "metrics: give each backoff config its own histogram label" matches the main change. The PR replaces the shared metrics.BackoffHistogramEmpty observer with four distinct observers. Each one is bound to its own label on TiKVBackoffHistogram, which is the core of the change. The title is specific and concise, and it fits the linked issue about the unnamed "Value" series.

Full details: Linked Issues check

Explanation

Issue #1802 asks to improve the Grafana label of the backoff histogram series shown as "Value" (the empty type label), possibly with reasonable names. The diff from merge base to head meets this. In metrics/shortcuts.go, BackoffHistogramEmpty (TiKVBackoffHistogram.WithLabelValues("")) is replaced by four observers labeled txnNotFound, maxTsNotSynced, commitTSLag, and regionNotInitialized. In config/retry/config.go, BoTxnNotFound, BoMaxTsNotSynced, BoCommitTSLag, and BoMaxRegionNotInitialized now bind to these observers. Their backoff parameters and errors are unchanged. The PR adds TestBackoffConfigsHaveNamedMetric in config/retry/config_test.go. I read the test only through the AI-generated summary, not its source. The diff confirms the code change. I did not search the whole repository for other uses of BackoffHistogramEmpty. The PR author reports none, so the build risk is unverified but does not affect the requirement.

Full details: Out of Scope Changes check

Explanation

All changed hunks in the inspected diffs serve the #1802 objective. metrics/shortcuts.go removes BackoffHistogramEmpty and adds four named observers. config/retry/config.go rebinds only the four configs that used the empty-label observer. The added test in config/retry/config_test.go checks the new labels and the absence of an empty type label, so it supports the objective. The removal of the exported BackoffHistogramEmpty is an API change. The PR description discloses it, and it follows directly from the objective, so it is not unrelated scope.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Repository UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 0296b987-3794-4900-b5bc-775467deefd5
Commits

Reviewing files that changed from the base of the PR and between 0e38031 and f251e7b.

Files selected for processing: 3

  • config/retry/config.go
  • config/retry/config_test.go
  • metrics/shortcuts.go

Included review availability: This review used your included allowance. Your plan provides up to 2 included reviews per hour; 1 remain after this review.

@ti-chi-bot ti-chi-bot Bot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Nov 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

contribution This PR is from a community contributor. dco-signoff: yes Indicates the PR's author has signed the dco. first-time-contributor Indicates that the PR was contributed by an external member and is a first-time contributor. needs-ok-to-test Indicates a PR created by contributors and need ORG member send '/ok-to-test' to start testing. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BackoffHistogramEmpty is displayed as confusing "Value" in grafana

1 participant