Skip to content

[roadmap] Reviewer model selection — tier by stakes, resolve by host #89

Description

@azevedo

Reviewer subagents hardcode their model in frontmatter with no override path. That is invisible to the user, unreachable from the selection ledger, and unresolvable on a non-Anthropic host.

The pin

Ten agents pin a model; the other eight inherit:

  • model: sonnet — all 8 reviewers (architecture, security, simplification, error-handling, test-coverage, deep-module, complexity, comment-quality) plus research-analyzer, research-locator
  • model: inheritcodebase-*, convention-checker, interface-design-generator, learnings-researcher, repo-researcher, spec-flow-analyzer

skills/ba-review/SKILL.md:463 dispatches via subagent_type: dev-workflow:<name> and passes no model, so frontmatter wins on every host.

The pin is an artifact, not a decision. docs/plans/2026-05-03-refactor-decouple-refactor-advisor-plan.md:24 records model: sonnet as the "canonical shape" for review agents — a frontmatter normalization pass, never a cost or capability judgment. No CI check pins it.

Three separable problems

Problem Reported by
P1 No override path — reviewers cannot follow the host's configured model external user report
P2 An 8-way fan-out default must stay cheap on large diffs maintainer usage
P3 Corroboration math treats all agreement as independent filed separately as #90

P1 in the field: a user running the plugin on Cursor with Grok selected reported Sonnet being dispatched — paying for a model they did not choose, on a host where sonnet is not even the capable option they configured.

P2 is why "just use model: inherit" is wrong. It hands an 8-way fan-out to whatever the session happens to be. The default must stay cheap and deliberate.

Already recorded as a residual: #55 shipped noting "the evidence corpus was collected at model: inherit while it ships model: sonnet" — the discrepancy has been known and unaddressed since v0.45.0.

Sketch (for the brainstorm to accept or reject)

Model becomes a property of a reviewer tier; the tier is deterministic (frontmatter-derivable, never per-run judgment) and visible in the selection ledger.

  • deepsecurity-reviewer, error-handling-reviewer. Inherits the session model, no override.
  • standard — the other six. Pinned cheap default.

Resolution order:

deep      -> inherit session model
standard  -> pinned default; if the name does not resolve on this host,
             omit the override and inherit
--model X -> forces both tiers to X (per-run escape hatch)
config    -> optional per-checkout default

The fallback clause is what quietly fixes P1: on a non-Anthropic host sonnet does not resolve, so everything inherits — no flag and no config file needed. On Claude Code nothing changes.

The tier line is drawn on whether anything downstream catches the miss, not on repair cost. A bad abstraction gets caught by the next person who reads it; a missing auth check does not. That distinction matters: a repair-cost framing would pull complexity-reviewer into the deep tier, recoverability does not.

Prior art

  • docs/research/2026-07-21-autoreview-skill-vs-ba-review-research.md — already names this: reviewer identity is "Same model, fanned out into 7 built-in + discovered subagents, each with a lens not a different brain", and autoreview's whole value proposition is an independent model.
  • compound-engineering ce-code-review, references/dispatch-reviewers.md § Model tiering — three highest-stakes personas inherit the session model, everything else takes an explicit mid-tier override. Their framing: "Omitting the override on a top-tier parent session silently runs that reviewer at the expensive tier — the regression this prevents." Note the direction: inherit is their default, cheap is the override. Ours is inverted — and their defensive direction exists because their tier list is prompt-carried and droppable, which a frontmatter field is not. We can point the default at cheap and still be safe.
  • Their portability rule is worth lifting verbatim: where the host's dispatch primitive exposes no model selector, "a working review on the parent model beats a broken dispatch on an unrecognized name."

Overlaps

Open questions for the brainstorm

  1. Is the deep tier 2 reviewers, or does architecture-reviewer belong there too?
  2. Does the tier live in agent frontmatter (deterministic, but a new field) or in the skill's roster table (one file, but prompt-carried)?
  3. --model flag, config key, or both? A per-checkout default is what an on-a-different-host user actually needs; a flag is what a per-run escalation needs.
  4. Does this belong to /ba-review only, or also /ba-review-plan (7 reviewers) and /ba-research (2 pinned research agents)?

The correctness-lens question is already owned by #6

An earlier revision of this issue flagged "there is no correctness lens on the roster" as possibly needing its own item. It does not — #6 owns it. That issue's Context section states the gap exactly ("The context flows in; no completeness verdict flows out") and its Decision already settled the shape: a ## Requirements Completeness section in the consolidated output, explicitly "Not implemented as an 8th reviewer agent." So the roster question is answered; do not reopen it here.

Two notes for whoever picks this up:

Blast radius

--model parsing in both review skills · ledger gains a column (never-hide mirrors in README.md + CLAUDE.md) · tier field across 8 agent files · the eight/seven reviewer-count literals if the roster presentation changes. Nothing here is CI-pinned, so it is another hand-maintained mirror set.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    cluster:model-fitFit dev-workflow's prompt surface to current-generation model behaviorcluster:review-quality/ba:review & review-plan qualityneeds-brainstormIdea not yet shaped into a plan

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions