Skip to content

[roadmap] Pin the protected-artifacts guard per dispatch block, not just the confidence literal #82

Description

@azevedo

Continuation of #64, which shipped its stated three checks and closed correctly. This is a fourth check in the same family: deterministic, grep-level, no model in the loop. Distinct from #7 (behavioral testing of reviewer prompts), which stays declined.

Problem

rubric-mirror asserts one contract per dispatch block — the confidence literal N ∈ {0, 25, 50, 75, 100}. The protected-artifacts guard is pinned nowhere, and it is the other half of what was lost in the original incident (see docs/solutions/prompt-authoring/2026-08-09-per-dispatch-block-ci-catches-template-drift.md).

That gap is not hypothetical. Measured on the branch that fixed the first half, one of three ba-review-plan templates carried the literal but not the guard, while the file asserted in prose that all three carried both. Closed by hand in bc842fa; nothing stops it recurring.

No file under agents/ contains the guard either, so the built-in reviewers have no fallback. CLAUDE.md calls the guard load-bearing for /ba-review-plan because the reviewed plan itself lives under docs/plans/.

Scope

Generalize rubricMirrorCheck from a single literal to a contract table, and run both existing loops (per-occurrence and per-block) over it.

const DISPATCH_BLOCK_CONTRACTS = [
  { id: 'confidence-set',
    literal: 'N ∈ {0, 25, 50, 75, 100}',
    anySpelling: /N\s*\s*\{[^}]*\}/ },
  { id: 'protected-artifacts',
    literal: 'Do not suggest deleting, relocating, renaming, or otherwise changing the existence or path of any file under `docs/brainstorms/`, `docs/plans/`, `docs/solutions/`, `docs/research/`, or `docs/reviews/`',
    anySpelling: /docs\/(brainstorms|plans|reviews)\// },
];

The non-obvious part

The guard is line-wrapped differently in the two files — one 426-char line in ba-review, four ~140-char lines in ba-review-plan. A per-line .includes() finds it in one file and not the other. The block body must be joined and whitespace-collapsed before the byte-exact comparison:

// ba-review keeps the guard on one line; ba-review-plan wraps it across four. Byte-exact on a
// whitespace-collapsed join, not on any single line — otherwise the check passes on one file's
// formatting and fails on the other's for no semantic reason.
const flat = body.join(' ').replace(/\s+/g, ' ');

This weakens the byte-exactness the check exists for. It is the right trade (wrap position carries no meaning), but the reason must be recorded at the call site or a later author will restore strict comparison and CI will fail on formatting.

Fixtures

buildRubricTree already accepts planTaskBlocks, so these are cheap. Each must be A/B-verified against the pre-change checker — a fixture that passes on both versions pins nothing.

  • block keeps the literal, loses the guard → FAIL
  • guard present but drops one protected root → FAIL
  • guard line-wrapped across lines → PASS (catches over-firing normalization)

Open scope call

Whether to stop at the guard or extend the table to the other two items templates carry — native ## Must Address vocabulary, most-specific-key anchoring. Neither is a parser contract, so pinning them byte-exact may be over-specification under the trust gradient. Decide before implementing, not during.

Estimate

~45 min. Three edits to scripts/check-invariants.mjs plus three fixtures in scripts/selfcheck-invariants.mjs.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    cluster:infraPlugin plumbing / namespacingreadyClear starting point — can build now

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions