feat(compliance): collection fixture preparation and output contract by bokelley · Pull Request #8376 · adcontextprotocol/adcp

bokelley · 2026-10-21T15:52:35Z

Defines the protocol side of #8254: how a storyboard selects a real collection-bearing product from the seller's catalog, captures its actual selectors and pricing from the same resolved graph, and how the runner reports preparation failures. The SDK implementation is adcontextprotocol/adcp-client#3354. The reporting storyboards that use it are held in #8336. #8259 states the applicability prerequisites.

Two files: storyboard-schema.yaml (authoring grammar) and runner-output-contract.yaml (output contract).

What it adds

Authoring (storyboard-schema.yaml)

collection_selection_satisfies: a closed fixture_resolution match operator (collection_targeting: fixed|selectable|any, minimum_pairs: 1|2, require_cross_publisher_collection). Products are qualified before deterministic ordering, so a nonqualifying earlier product can't hide a later qualifying one. It adds no new seller call and no synthetic fallback.
Parent product selection excludes products that lack a matching scoped price for each dependent discover-only pricing declaration. This is an existence check, not an assignment solver.
context_outputs on products and pricing options, in three exclusive forms: path, snapshot: selected_entity (keeps optional commercial fields like min_spend_per_package absent instead of fabricating 0) and seven named witness values (strict_subset, cross_publisher_relation, negatives, …). Captures publish only after the whole graph resolves.
Reserved __adcp_fixture_ context namespace with atomic, freeze-on-commit writes. __proto__ / constructor / prototype are forbidden context keys.
Reusable preparation is pinned to graph, storyboard, schema artifact, route, brand/account/sandbox and credential scope. A live host authorizer runs on every lookup, after effective account and credential resolution and before any contact.
collection_set_equals_context: an authored check comparing domain-qualified (publisher_domain, collection_id) pair sets, ignoring grouping and order.
Downgrade: a runner lacking the operator fails at load. That's an authoring/runner incompatibility, never seller nonsupport.

Output (runner-output-contract.yaml)

Synthesized check kinds fixture_capture_unbuildable (3 stages), protected_context_violation and prepared_fixtures_authorization, each with an exact expected/actual and json_pointer basis, plus a request/response rule: real records when an exchange happened, explicit null only when there was genuinely no contact.
Authorization refusal is a host-boundary denial (maintainer decision). The refused caller gets a typed authorization error (an API error when hosted, a typed error in the library) carrying only the refusal code. It gets no grading result and no partial or redacted projection. The complete, unredacted result, including fixture_resolutions with seller_ids, is retained in the host's authorized audit scope, and every existing output MUST applies to that record. This settles the conflict between the evidence MUSTs and nondisclosure without a waiver and without fabricated or omitted records.

Questions for reviewers

Global __adcp_fixture_ reservation and forbidden prototype keys: this applies to all storyboards, not just ones using the feature. No published storyboard uses the prefix today, but confirm we're comfortable with a minor bump.
Last-witness association: when several direct collection predicates are authored, the last eligible one supplies the witness. Witnesses aren't merged. The recommended authoring is one predicate per declaration.
Parent exclusion for dependent scoped pricing changes which product is selected in an edge case that previously failed later. Selection order is unchanged otherwise.
request.payload basis (open in the SDK review): does it mean the bytes actually sent on the wire, or the post-substitution payload before transport encoding? The SDK behavior is unchanged and predates this work. Raised here for a ruling.
Possible follow-up: a sub-reason discriminator for the three fixture_capture_unbuildable stages. Not implemented in the SDK and not needed for this PR.

The cross-step rules for context_key echo and context_key_absent now state the collection_set_equals_context exceptions explicitly (it echoes its key, and an absent key FAILS because fixture captures publish all-or-nothing), so a third-party runner reading those rules alone can't grade a missing fixture as a pass.

The SDK reference implementation (adcontextprotocol/adcp-client#3354) implements this contract, including Option 1 (PreparedFixturesDeniedError plus an audit sink).

Refs #8254, #8259, #8336.

🤖 Generated with Claude Code

aao-secretariat

Ladon verdict: Approve

Approve — additive, prose-only compliance contract for collection fixture preparation.

Checked:

Changeset: correct minor on adcontextprotocol for a non-breaking addition — not patch/minor shipping a breaking change, not a hand-edited version.
Schema↔docs coherence: cross-file coherence verified across storyboard-schema.yaml (authoring grammar) and runner-output-contract.yaml (output contract) — three synthesized check kinds, authored collection_set_equals_context check + required context_key, seven named witnesses, and the "four check kinds" schema_id/schema_url null rule all consistent.
oneOf discriminators: no oneOf/schema-shape changes introduced.
Fail-closed __adcp_fixture_ reservation with forbidden prototype keys and the authorization-refusal/retained-record rule verified.
Classification: additive/Normative (non-breaking) — Breaking-class hard gate does not apply, so the human-ratification escalation requirement is not triggered.

No Critical/High/Medium findings. high_risk false; gated_paths false; no no-auto-approve team match. Decision table rows 1–8 do not fire → row 9 approve.

Authoring grammar (storyboard-schema.yaml) and runner output contract (runner-output-contract.yaml) for collection-bearing fixture selection, same-graph captures, protected fixture context, and the synthesized preparation/authorization outputs. Authorization refusals are host-boundary denials; the complete result is retained in the authorized audit scope. Refs #8254, #8259. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

aao-secretariat

Ladon verdict: Approve

Approve — compliance-contract files are cross-file coherent with no blocking issues.

Reviewed this subsequent pass on an identical diff (3 files, 329/-2) that previously approved clean. Checked:

Synthesized check kinds, the new collection_set_equals_context authored check, the context_key output field, and the seven named witnesses all match across storyboard-schema.yaml and runner-output-contract.yaml.
The storyboard "required context_key" vs output-contract "missing context_key" case is deliberately reconciled by the branch-safe-exclusion rule — not a contradiction.
Changeset present (minor), appropriate scope. No static/schemas/source/** wire change, no dist/** edits. Neither file exceeds 200 net lines.
No oneOf discriminator regression, no immutable-artifact modification, no governance-precedent conflict.

No Critical/High/Medium findings. gated_paths false; review_decision APPROVED; no no-auto-approve team match; high_risk false. None of decision rows 1–8 fire → row 9 approve.

Implements Option 1 of `refusal_disclosure_and_accounting` in the protocol amendment (adcontextprotocol/adcp#8376). An authorization failure is an ACCESS-CONTROL error, not a grading outcome. `runStoryboard` and `runStoryboardStep` now THROW `PreparedFixturesDeniedError` when a gate refuses. The error carries `refusalCode` — one of the six `PreparedFixturesRefusalCode` values — and nothing else: no grading result, partial or redacted, and therefore no fixture identities, `seller_ids`, captures, binding values, request/response records, or results from steps that ran before the grant was revoked. A hosted route maps it to a 403. The evidence still has to exist somewhere accountable, so the runner builds the COMPLETE, unredacted record first and hands it to the new `prepared_fixtures_audit_sink` option before throwing. That record keeps the synthesized `prepared_fixtures_authorization` check, the reserved captures, `fixture_resolutions` with `seller_ids`, and exact request and response records for every step that ran. Every existing output MUST applies to it. Configure no sink and the record is produced to nobody. A sink that throws does not change the outcome, and its own failure is never reported to the caller — that would be an oracle over host state. In the retained record the refusal check uses explicit `request: null`, `response: null`, `json_pointer: null`, `duration_ms: 0`. Explicit, not omitted: a missing key cannot be told apart from a runner that forgot to record one. `ValidationResult.request` / `.response` widen to `| null` to carry that. This REMOVES the `[redacted]` refusal projection entirely — the projection, the protected-value index, the substring matcher and the hint/validation scrubbing all go with it. That code existed to make a report safe to hand a refused caller; under this contract there is no report to hand them, so the whole disclosure class is gone by construction rather than by redaction. Nothing in the SDK redacts on the refusal path, and no marker may stand in for required evidence. `fixture_resolutions` is no longer withheld from a refused run's record, which removes the divergence from the contract's "required whenever the storyboard declares `fixture_resolution`" MUST without needing a carve-out. Also: the CodeQL suppression for `fingerprintCredential`, matching the five existing `js/insufficient-password-hash` suppressions — the fingerprint is an in-process equality tag under a per-process random key, not a password verifier, and a work-factor KDF would add latency to every step for no security gain. Maintainer-dispositioned. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…it at the cross-step rules The general cross-step rules say context_key is not echoed and an absent key passes with a context_key_absent observation. State both exceptions where a third-party runner would read them. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

aao-secretariat

Ladon verdict: Approve

Approve — clean spec clarification, no blocking or medium findings.

This is a subsequent push after a prior clean approve. The delta is 40 lines across two compliance-source files, adding two symmetric Exception clauses for collection_set_equals_context: it echoes its authored context_key into validation_result (carve-out from the general no-echo rule) and MUST FAIL on an absent context_key (carve-out from the branch-safe context_key_absent success rule). Cross-references resolve correctly against collection_fixture_preparation_output and the full-PR text. Coherent, no spec drift, minor changeset present.

Checked: schema↔docs coherence (no wire-shape divergence), changeset scope/type (minor changeset present and appropriate), no oneOf discriminator regression, no released-artifact mutation (only source/* touched), not a Breaking-class change. No governance precedent contradicted.

Gate check: gated_paths is false, so the review_decision=REVIEW_REQUIRED gate (row 2) does not fire. No no-auto-approve team match. high_risk is false. No findings at any tier → falls through to row 9.

Qualify a real collection-bearing product before deterministic selection, capture the selected product / scoped price / collection witnesses from that same resolved graph, protect those captures behind an atomic write funnel, and make fixture preparation reusable across the full and standalone runners behind a host authorization boundary. Five additions: - `collection_selection_satisfies` — a bounded fixture match operator. `collection_targeting: fixed` accepts literal `false` OR the schema's omitted-false default; `selectable` requires literal `true`. `minimum_pairs: 1 | 2` keeps default materialization and strict-subset selection qualifying independently. `require_cross_publisher_collection` qualifies the declared carriage relation across every selector. Qualification runs BEFORE deterministic ordering, so a nonqualifying earlier product cannot conceal a later qualifying one. - Declaration-local `context_outputs` on `fixture_resolution` — `path:` resolves against the selected entity, `witness:` binds a bounded derived capture from the same evaluation, and `snapshot: selected_entity` captures the whole pinned entity so an optional commercial field can be legally absent without failing preparation or being gated behind a `present` predicate that would suppress a conformant seller. Nothing is defaulted and an explicit `0` is captured as `0`. Captures publish only after the whole graph resolves. - A protected `__adcp_fixture_` context namespace with one atomic write funnel for values, deletions, provenance and unavailable-output metadata. Every prospective write is validated before any mutation; committed values are deep-cloned, deep-frozen and handed out detached. - `prepareStoryboardFixtures()` — immutable prepared state both `runStoryboard` and `runStoryboardStep` accept via `prepared_fixtures`. Authority is re-checked on EVERY lookup through `prepared_fixtures_authorizer`, which is required for any reuse; omitting it fails closed with zero seller calls, for direct in-process state exactly as for an opaque reference. Authorization is enforced before any seller endpoint interaction at all — including MCP `initialize`, `tools/list`, capability discovery and controller seeding, not merely before the mutating call. - `collection_set_equals_context` — exact set comparison over atomic `(publisher_domain, collection_id)` pairs, through the same normalizer the predicate and witnesses use. A refusal is an ACCESS-CONTROL error, not a grading outcome (adcontextprotocol/adcp#8376, `refusal_disclosure_and_accounting` Option 1). `runStoryboard` / `runStoryboardStep` THROW `PreparedFixturesDeniedError` carrying only the machine refusal code — no grading result, partial or redacted, and so no fixture identities, `seller_ids`, captures, binding values, request/response records or earlier-step results. The complete, UNREDACTED record goes to the new `prepared_fixtures_audit_sink` first; configure no sink and it is produced to nobody. In that record the synthesized check carries explicit `request: null`, `response: null`, `json_pointer: null`, `duration_ms: 0` — explicit, because an omitted key cannot be told apart from a runner that forgot to record one. `ValidationResult.request` / `.response` widen to `| null` to carry it. Behaviour changes to audit before upgrading: 1. A `context` default, step `context_outputs` key, or `not_applicable_if[].context_keys` entry starting with `__adcp_fixture_`, or named `__proto__` / `constructor` / `prototype`, now fails at LOAD. Rename it. Audit with `grep -rnE '__adcp_fixture_|__proto__|constructor|prototype' your-storyboards/`. 2. Reuse that is refused now THROWS. If you `await runStoryboard(...)` with `prepared_fixtures` and branch on `result.overall_passed`, add a `catch` for `PreparedFixturesDeniedError` and map it to a 403. 3. Caller-supplied `context` / `context_provenance` / `response_derived_not_applicable_context_keys` carrying a reserved or prototype-sensitive key is refused identically from every entry point, where the full runner previously threw a bare `Error`. 4. Parent-product selection now skips a candidate that cannot satisfy a dependent discover-only pricing spec, so graphs that previously stranded resolve and a different product can win. Re-baseline if you assert on `seller_ids`. 5. `fixture_unsatisfied` is a canonical `RunnerSkipReason`, not a detailed reason mapped down to `not_applicable`, and both of the runner's unsatisfied paths now spell its detail the same way. 6. `context_outputs[].path` is validated through the resolver: a leading `/`, an empty dotted segment, and anything else reducing to zero segments are refused at load — the last of those would otherwise capture the whole entity under a scalar's key. Output evidence follows `runner-output-contract.yaml`'s `required_on_failure`: `schema_id` / `schema_url` are explicit nulls at all eight synthesized emission sites and on the authored check, and `fixture_capture_unbuildable` carries the real `get_products` / `comply_test_controller` exchange that produced the entity — a post-wire failure must not read as no contact. When discovery paginated, the recorded exchange is the page that actually carried the pinned product. Reviewed under REVIEW-STACKS: four Claude experts in parallel — the capture write funnel and prepared-state lookup authority SEPARATELY, plus protocol and code review — and three second-model Codex passes. Nine defects were found and fixed, five of them ship-blockers, each reproduced before the fix and each covered by a regression verified to fail without it: a route binding that let a graph pinned against one seller be reused against another; `path: "[]"` capturing the whole entity, uredacted; and three disclosure paths through the then-existing refusal projection, which Option 1 subsequently removed by construction. Six names are not yet in the published AdCP contracts and are registered by adcontextprotocol/adcp#8376, together with one normative exception: `collection_set_equals_context` must FAIL on an absent context key, against the cross-step `context_key_absent` rule, because fixture captures publish all-or-nothing and grading that green would certify the substitution the check exists to detect. Adopter guide: docs/guides/COLLECTION-FIXTURE-FOUNDATION.md Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…3354) * feat(testing)!: collection fixture foundation for storyboard runs Qualify a real collection-bearing product before deterministic selection, capture the selected product / scoped price / collection witnesses from that same resolved graph, protect those captures behind an atomic write funnel, and make fixture preparation reusable across the full and standalone runners behind a host authorization boundary. Five additions: - `collection_selection_satisfies` — a bounded fixture match operator. `collection_targeting: fixed` accepts literal `false` OR the schema's omitted-false default; `selectable` requires literal `true`. `minimum_pairs: 1 | 2` keeps default materialization and strict-subset selection qualifying independently. `require_cross_publisher_collection` qualifies the declared carriage relation across every selector. Qualification runs BEFORE deterministic ordering, so a nonqualifying earlier product cannot conceal a later qualifying one. - Declaration-local `context_outputs` on `fixture_resolution` — `path:` resolves against the selected entity, `witness:` binds a bounded derived capture from the same evaluation, and `snapshot: selected_entity` captures the whole pinned entity so an optional commercial field can be legally absent without failing preparation or being gated behind a `present` predicate that would suppress a conformant seller. Nothing is defaulted and an explicit `0` is captured as `0`. Captures publish only after the whole graph resolves. - A protected `__adcp_fixture_` context namespace with one atomic write funnel for values, deletions, provenance and unavailable-output metadata. Every prospective write is validated before any mutation; committed values are deep-cloned, deep-frozen and handed out detached. - `prepareStoryboardFixtures()` — immutable prepared state both `runStoryboard` and `runStoryboardStep` accept via `prepared_fixtures`. Authority is re-checked on EVERY lookup through `prepared_fixtures_authorizer`, which is required for any reuse; omitting it fails closed with zero seller calls, for direct in-process state exactly as for an opaque reference. Authorization is enforced before any seller endpoint interaction at all — including MCP `initialize`, `tools/list`, capability discovery and controller seeding, not merely before the mutating call. - `collection_set_equals_context` — exact set comparison over atomic `(publisher_domain, collection_id)` pairs, through the same normalizer the predicate and witnesses use. A refusal is an ACCESS-CONTROL error, not a grading outcome (adcontextprotocol/adcp#8376, `refusal_disclosure_and_accounting` Option 1). `runStoryboard` / `runStoryboardStep` THROW `PreparedFixturesDeniedError` carrying only the machine refusal code — no grading result, partial or redacted, and so no fixture identities, `seller_ids`, captures, binding values, request/response records or earlier-step results. The complete, UNREDACTED record goes to the new `prepared_fixtures_audit_sink` first; configure no sink and it is produced to nobody. In that record the synthesized check carries explicit `request: null`, `response: null`, `json_pointer: null`, `duration_ms: 0` — explicit, because an omitted key cannot be told apart from a runner that forgot to record one. `ValidationResult.request` / `.response` widen to `| null` to carry it. Behaviour changes to audit before upgrading: 1. A `context` default, step `context_outputs` key, or `not_applicable_if[].context_keys` entry starting with `__adcp_fixture_`, or named `__proto__` / `constructor` / `prototype`, now fails at LOAD. Rename it. Audit with `grep -rnE '__adcp_fixture_|__proto__|constructor|prototype' your-storyboards/`. 2. Reuse that is refused now THROWS. If you `await runStoryboard(...)` with `prepared_fixtures` and branch on `result.overall_passed`, add a `catch` for `PreparedFixturesDeniedError` and map it to a 403. 3. Caller-supplied `context` / `context_provenance` / `response_derived_not_applicable_context_keys` carrying a reserved or prototype-sensitive key is refused identically from every entry point, where the full runner previously threw a bare `Error`. 4. Parent-product selection now skips a candidate that cannot satisfy a dependent discover-only pricing spec, so graphs that previously stranded resolve and a different product can win. Re-baseline if you assert on `seller_ids`. 5. `fixture_unsatisfied` is a canonical `RunnerSkipReason`, not a detailed reason mapped down to `not_applicable`, and both of the runner's unsatisfied paths now spell its detail the same way. 6. `context_outputs[].path` is validated through the resolver: a leading `/`, an empty dotted segment, and anything else reducing to zero segments are refused at load — the last of those would otherwise capture the whole entity under a scalar's key. Output evidence follows `runner-output-contract.yaml`'s `required_on_failure`: `schema_id` / `schema_url` are explicit nulls at all eight synthesized emission sites and on the authored check, and `fixture_capture_unbuildable` carries the real `get_products` / `comply_test_controller` exchange that produced the entity — a post-wire failure must not read as no contact. When discovery paginated, the recorded exchange is the page that actually carried the pinned product. Reviewed under REVIEW-STACKS: four Claude experts in parallel — the capture write funnel and prepared-state lookup authority SEPARATELY, plus protocol and code review — and three second-model Codex passes. Nine defects were found and fixed, five of them ship-blockers, each reproduced before the fix and each covered by a regression verified to fail without it: a route binding that let a graph pinned against one seller be reused against another; `path: "[]"` capturing the whole entity, uredacted; and three disclosure paths through the then-existing refusal projection, which Option 1 subsequently removed by construction. Six names are not yet in the published AdCP contracts and are registered by adcontextprotocol/adcp#8376, together with one normative exception: `collection_set_equals_context` must FAIL on an absent context key, against the cross-step `context_key_absent` rule, because fixture captures publish all-or-nothing and grading that green would certify the substitution the check exists to detect. Adopter guide: docs/guides/COLLECTION-FIXTURE-FOUNDATION.md Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore(testing): drop unused helper and imports flagged by code quality Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>

aao-secretariat Bot approved these changes Oct 21, 2026

View reviewed changes

bokelley force-pushed the bokelley/collection-fixture-output-contract branch from 21005e7 to 5b02991 Compare October 21, 2026 16:05

aao-secretariat Bot previously approved these changes Oct 21, 2026

View reviewed changes

bokelley dismissed aao-secretariat[bot]’s stale review via b23a9b2 October 22, 2026 00:24

aao-secretariat Bot approved these changes Oct 22, 2026

View reviewed changes

bokelley merged commit 277e0a5 into main Oct 22, 2026
55 checks passed

bokelley deleted the bokelley/collection-fixture-output-contract branch October 22, 2026 12:20

Navigation Menu

Repository navigation

feat(compliance): collection fixture preparation and output contract - #8376

feat(compliance): collection fixture preparation and output contract#8376
bokelley merged 2 commits into
mainfrom
bokelley/collection-fixture-output-contract

bokelley commented Oct 21, 2026 •

edited

Loading

Uh oh!

aao-secretariat Bot left a comment

Uh oh!

aao-secretariat Bot left a comment

Uh oh!

aao-secretariat Bot left a comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

bokelley commented Oct 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What it adds

Questions for reviewers

Uh oh!

aao-secretariat Bot left a comment

Choose a reason for hiding this comment

Ladon verdict: Approve

Uh oh!

aao-secretariat Bot left a comment

Choose a reason for hiding this comment

Ladon verdict: Approve

Uh oh!

aao-secretariat Bot left a comment

Choose a reason for hiding this comment

Ladon verdict: Approve

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

bokelley commented Oct 21, 2026 •

edited

Loading