docs(reports): restructure by tab and document the Impact report - #286
docs(reports): restructure by tab and document the Impact report#286chris-absmartly wants to merge 2 commits into
Conversation
Experiment reports are presented as three tabs (Velocity, Decisions, Impact) but the page listed Decisions overview and Decisions history as top-level sections alongside the velocity report, so the tab structure was not visible in the outline. Restructure the page to match the UI: - one H2 per tab, with Decisions overview and Decisions history demoted to H3 under Decisions report - merge the Permissions block and the decision-types paragraph that were duplicated verbatim across both decisions sections - add a tab summary table at the top of the page The Decisions overview heading keeps its text so the #decisions-overview anchor, linked from the March and April 2025 release notes, still resolves. Document the new Impact report: metric selection, depreciation, total estimated impact, full-on experiments, impact by experiment and impact over time. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
WalkthroughThe experiment reports documentation now uses three tabs: Velocity, Decisions, and Impact. It adds shared settings and filters. It reorganises Decisions into overview and history sections. It documents Impact eligibility, permissions, metrics, depreciation, estimated impact, confidence ranges, experiment breakdowns, forecasts, and chart controls. Estimated code review effort: 3 (Moderate) | ~20 minutes Possibly related PRs
Suggested reviewers: Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
docs/web-console-docs/experiments/Experiment-reports.mdx (1)
128-180: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winCorrect the decision-type grammar.
Line 131 says “Possible decisions types”. Change it to “Possible decision types”.
Proposed fix
-Possible decisions types included in the report are `Full on` where a tested change is fully rolled out with or without full supporting evidence; +Possible decision types included in the report are `Full on` where a tested change is fully rolled out with or without full supporting evidence;🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/web-console-docs/experiments/Experiment-reports.mdx` around lines 128 - 180, In the Decisions report introduction, update the phrase “Possible decisions types” to “Possible decision types” while leaving the surrounding content unchanged.Source: Linters/SAST tools
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/web-console-docs/experiments/Experiment-reports.mdx`:
- Around line 182-194: Correct the Decisions history documentation: change
“overtime” to “over time” in the timeline description, add the missing object
after “allows” in the Decision type filter sentence, and replace “rational” with
“rationale” in the Decision card description.
---
Outside diff comments:
In `@docs/web-console-docs/experiments/Experiment-reports.mdx`:
- Around line 128-180: In the Decisions report introduction, update the phrase
“Possible decisions types” to “Possible decision types” while leaving the
surrounding content unchanged.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: b712a3ba-c199-43d8-925e-54e297bba88e
⛔ Files ignored due to path filters (5)
static/img/experiment-report/impact-depreciation-settings.pngis excluded by!**/*.pngstatic/img/experiment-report/impact-over-time-tooltip.pngis excluded by!**/*.pngstatic/img/experiment-report/impact-over-time.pngis excluded by!**/*.pngstatic/img/experiment-report/impact-report.pngis excluded by!**/*.pngstatic/img/experiment-report/impact-total-estimated-impact.pngis excluded by!**/*.png
📒 Files selected for processing (1)
docs/web-console-docs/experiments/Experiment-reports.mdx
| ### Decisions history | ||
|
|
||
| The `Decisions history` provides a timeline of decisions overtime. | ||
| Possible decisions types included in the report are `Full on` where a tested change is fully rolled out with or without full supporting evidence; | ||
| `Keep current` where the tested change is not rolled out, and the existing experience remains; | ||
| and `Abort` where it was decided to stop the experiment before it could provide reliable evidence. | ||
|
|
||
| This report can be used to browse through past decisions to understand the reasoning behind each of them. | ||
|
|
||
| #### Filter | ||
|
|
||
| The `Decision type` filter allows to select the type of decisions to show on the timeline. | ||
|
|
||
| #### Decision card | ||
|
|
||
| Each decision card provides an overview of the past decisions, highlighting the hypothesis, the rational behind the decision and the key metrics supporting the decision. |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Correct the Decisions history text.
Line 184 should say “over time”. Line 190 needs an object after “allows”. Line 194 uses “rational” instead of “rationale”.
Proposed fix
-The `Decisions history` provides a timeline of decisions overtime.
+The `Decisions history` provides a timeline of decisions over time.
-The `Decision type` filter allows to select the type of decisions to show on the timeline.
+The `Decision type` filter lets you select the decision types to show on the timeline.
-Each decision card provides an overview of the past decisions, highlighting the hypothesis, the rational behind the decision and the key metrics supporting the decision.
+Each decision card provides an overview of past decisions, highlighting the hypothesis, the rationale behind the decision, and the key metrics that support the decision.📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| ### Decisions history | |
| The `Decisions history` provides a timeline of decisions overtime. | |
| Possible decisions types included in the report are `Full on` where a tested change is fully rolled out with or without full supporting evidence; | |
| `Keep current` where the tested change is not rolled out, and the existing experience remains; | |
| and `Abort` where it was decided to stop the experiment before it could provide reliable evidence. | |
| This report can be used to browse through past decisions to understand the reasoning behind each of them. | |
| #### Filter | |
| The `Decision type` filter allows to select the type of decisions to show on the timeline. | |
| #### Decision card | |
| Each decision card provides an overview of the past decisions, highlighting the hypothesis, the rational behind the decision and the key metrics supporting the decision. | |
| ### Decisions history | |
| The `Decisions history` provides a timeline of decisions over time. | |
| This report can be used to browse through past decisions to understand the reasoning behind each of them. | |
| #### Filter | |
| The `Decision type` filter lets you select the decision types to show on the timeline. | |
| #### Decision card | |
| Each decision card provides an overview of past decisions, highlighting the hypothesis, the rationale behind the decision, and the key metrics that support the decision. |
🧰 Tools
🪛 LanguageTool
[grammar] ~190-~190: Did you mean “selecting”? Or maybe you should add a pronoun? In active voice, ‘allow’ + ‘to’ takes an object, usually a pronoun.
Context: ...lter The Decision type filter allows to select the type of decisions to show on the ti...
(ALLOW_TO)
[grammar] ~190-~190: In this context, ‘type’ should agree in number with the noun after ‘of’.
Context: ...ision type` filter allows to select the type of decisions to show on the timeline. #### Decision...
(TYPE_OF_PLURAL)
[misspelling] ~194-~194: Did you mean “rationale” (=an explanation / underlying reason)?
Context: ...sions, highlighting the hypothesis, the rational behind the decision and the key metrics...
(RATIONAL_RATIONALE)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/web-console-docs/experiments/Experiment-reports.mdx` around lines 182 -
194, Correct the Decisions history documentation: change “overtime” to “over
time” in the timeline description, add the missing object after “allows” in the
Decision type filter sentence, and replace “rational” with “rationale” in the
Decision card description.
Source: Linters/SAST tools
✅ Deploy Preview for absmartly-docs ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
Recapture the impact report and total estimated impact screenshots after the UI copy fixes: the inaccurate "Primary metric totals shown. Expand rows..." subtitle is gone and the test-data prefixes are stripped from the experiment names. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/web-console-docs/experiments/Experiment-reports.mdx`:
- Around line 299-301: Update the chart-control descriptions near the Cumulative
/ Daily, Day / Week / Month / Quarter / Year, and Forecast entries to use
complete, correctly inflected verbs, while preserving their existing meanings
and option details.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 8d4cae37-c84c-403b-9cb0-654a402d366b
⛔ Files ignored due to path filters (2)
static/img/experiment-report/impact-report.pngis excluded by!**/*.pngstatic/img/experiment-report/impact-total-estimated-impact.pngis excluded by!**/*.png
📒 Files selected for processing (1)
docs/web-console-docs/experiments/Experiment-reports.mdx
| - **Cumulative / Daily** switch between total impact accumulated to date and the impact contributed on each individual day | ||
| - **Day / Week / Month / Quarter / Year** the granularity at which points are plotted | ||
| - **Forecast** project the trend forward by `1 month`, `3 months`, `6 months` or `1 year`, or turn the projection `Off` |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Use complete verbs for the chart-control descriptions.
Lines 299-301 use incomplete or incorrectly inflected verbs. This makes the documented behaviour less clear.
Proposed wording
-- **Cumulative / Daily** switch between total impact accumulated to date and the impact contributed on each individual day
-- **Day / Week / Month / Quarter / Year** the granularity at which points are plotted
-- **Forecast** project the trend forward by `1 month`, `3 months`, `6 months` or `1 year`, or turn the projection `Off`
+- **Cumulative / Daily**: Switches between total impact accumulated to date and the impact contributed on each individual day
+- **Day / Week / Month / Quarter / Year**: Sets the granularity at which points are plotted
+- **Forecast**: Projects the trend forward by `1 month`, `3 months`, `6 months` or `1 year`, or turns the projection `Off`📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| - **Cumulative / Daily** switch between total impact accumulated to date and the impact contributed on each individual day | |
| - **Day / Week / Month / Quarter / Year** the granularity at which points are plotted | |
| - **Forecast** project the trend forward by `1 month`, `3 months`, `6 months` or `1 year`, or turn the projection `Off` | |
| - **Cumulative / Daily**: Switches between total impact accumulated to date and the impact contributed on each individual day | |
| - **Day / Week / Month / Quarter / Year**: Sets the granularity at which points are plotted | |
| - **Forecast**: Projects the trend forward by `1 month`, `3 months`, `6 months` or `1 year`, or turns the projection `Off` |
🧰 Tools
🪛 LanguageTool
[duplication] ~299-~299: Possible typo: you repeated a word.
Context: ...e impact contributed on each individual day - Day / Week / Month / Quarter / Year the g...
(ENGLISH_WORD_REPEAT_RULE)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/web-console-docs/experiments/Experiment-reports.mdx` around lines 299 -
301, Update the chart-control descriptions near the Cumulative / Daily, Day /
Week / Month / Quarter / Year, and Forecast entries to use complete, correctly
inflected verbs, while preserving their existing meanings and option details.
1 participant
Restructure
Experiment reports are presented as three tabs — Velocity, Decisions, Impact — but the page listed
Decisions overviewandDecisions historyas top-level sections alongside the velocity report, so the tab structure wasn't visible in the outline or the sidebar TOC.The page now has one H2 per tab:
## Velocity report## Decisions report— with Decisions overview and Decisions history demoted to H3## Impact report— newAlso merged the
Permissionsblock and the decision-types paragraph, which were duplicated verbatim across both decisions sections.Anchor compatibility: the
Decisions overviewheading keeps its exact text, so#decisions-overview— linked from the March 2025 and April 2025 release notes — still resolves. The velocity heading rename does change its anchor, but nothing links to it.Impact report
New section covering the feature: metric selection, depreciation (the five presets and what they mean), total estimated impact, full-on experiments, impact by experiment, and the impact over time chart with its controls and series.
Five screenshots added under
static/img/experiment-report/.Note for review
The UI subtitle on the Impact by experiment table currently reads "Primary metric totals shown. Expand rows to see all metrics with confidence intervals." Both halves look inaccurate — the table shows the selected metric (rows tagged
Secondaryappear in it), and rows don't expand. The docs describe the actual behaviour rather than the subtitle; the product copy is being updated separately.yarn buildpasses withonBrokenLinks: "throw".🤖 Generated with Claude Code
Summary by CodeRabbit