fix(compliance): qualify independent catalog reporting scopes

.changeset/catalog-optional-delivery-reporting.md

@@ -0,0 +1,15 @@
+---
+"adcontextprotocol": patch
+---
+
+`advanced_delivery_reporting` no longer seeds a video product. It grades
+format-breakdown reporting only when your wholesale sandbox catalog contains a
+non-guaranteed, fixed-price CPM product with
+`reporting_capabilities.supports_format_breakdown: true`. When completed valid
+catalog discovery finds no qualifying product, it reports `not_applicable`
+(`fixture_unsatisfied`). Time-based-view checks move to a new
+`time_based_delivery_reporting` storyboard, selected independently from native
+wholesale sandbox products listing `time_based_views`, with the same
+non-guaranteed, fixed-price CPM restriction. Neither requires a specific threshold, basis, format
+kind or non-empty rows. Partially addresses #8254; collection storyboards are
+unchanged.

docs/reference/known-limitations.mdx

@@ -81,7 +81,12 @@ Request-specific configured offers are not a gap: canonical products carry no `i
 - **Training-agent CI is a strict regression gate, not a single conformance score.** Every in-scope, non-quarantined storyboard that runs against the reference agent must pass. The agent is split across protocol-specific tenants and is exercised against both current source and frozen compatibility bundles. Before each run, the matrix calls `get_adcp_capabilities` and resolves the mandatory universal, protocol, and specialism bundles from that live declaration; storyboards outside the resulting scope are reported as not applicable, while known whole-storyboard runner blockers remain separately quarantined. Passed, failed, skipped, step-level not-applicable, storyboard-level not-applicable, and whole-storyboard quarantines remain separate output fields because collapsing them into one fraction would hide both regressions and coverage gaps. Use the latest `Training Agent Storyboards` workflow run and its per-storyboard artifacts for current status. Whole-storyboard quarantines and step-level compatibility skips live next to the runner in `server/tests/manual/run-storyboards.ts`; each names the blocker and its removal condition, with upstream issue links where one exists. Step-level compatibility skips contribute to the ordinary skipped-step total rather than the storyboard quarantine column. Do not copy their counts into documentation.
 - **The remaining conformance gaps are classified, not hidden.** Current-source grading resolves missing `required_tools`, capability prerequisites, and opt-in runner contracts before executing stateful setup; optional evaluator and localization scenarios now carry executable capability gates instead of relying on descriptive `interaction_model` metadata. The creative lifecycle webhook scenario separately gates on the creative-library prerequisite, while seller notification support and the reference tenants' controller-driven transition remain named coverage limits rather than being inferred from library support. The security probe selects an advertised allowlisted read such as `list_accounts`. The current reference matrix still names exceptions for loopback webhook execution and controller-driven creative lifecycle transitions, governance delivery-check request shaping, and preservation of deprecated selectors alongside canonical selectors. Whole-storyboard blockers remain quarantined; narrow runner defects remain visible as skipped steps and are never counted as passed. Frozen compatibility bundles may retain additional named skips where an immutable older contract cannot express current behavior. Current source keeps the primary sales sequences ungoverned, exercises valid governance registration plus the single-agent constraint in a reusable capability-gated scenario, and reserves the full registration → approval → governed-create loop for governance-aware sellers. The reference seller performs the registered governance agent's online execution check before committing.
 - **Hosted verifier 3.2 profile: known operational boundaries.** The hosted conformance verifier at AgenticAdvertising.org runs a capability-gated storyboard suite against registered agents. Capability-excluded storyboards grade as not-applicable coverage gaps, never as failures ([#6907](https://github.com/adcontextprotocol/adcp/issues/6907)); storyboard test-kit credentials are prepared per storyboard rather than supplied as one run-level kit ([#6735](https://github.com/adcontextprotocol/adcp/issues/6735)); budget-limited heartbeats rotate their storyboard starting point for coverage over consecutive runs ([#6632](https://github.com/adcontextprotocol/adcp/issues/6632)); and refresh results expose `auth_available` so missing verifier credentials are distinguishable from target-agent conformance failure ([#7070](https://github.com/adcontextprotocol/adcp/issues/7070)). The external smoke check verifies discovery, storyboard resolution, and auth-failure classification before release.
-- **Some late 3.2 channel-measurement additions do not yet have end-to-end reference-agent storyboards.** The `advanced_delivery_reporting` storyboard covers `requested_metrics`, canonical `by_format` rows, applied sort echoes, truncation disclosure, and `time_based_views`; expanded OOH/audio measurement identities still have schema and focused contract-test coverage rather than one complete discovery → purchase → delivery journey per metric family. Audience activation has capability-gated storyboards for both discovery and execution: `audience_activation_discovery` covers the catalog-wide union, per-product declarations, vendor matching, OR/wildcard filter semantics, and exclusion of undeclared products; `external_audience_source_binding` binds a declared shared dataset, preserves its source and match state through discovery, uses the resulting audience in a media buy, and rejects undeclared rails or cross-transport updates. The reference seller deliberately advertises `dataset_query`, not `platform_distribution`, so the latter remains not applicable rather than simulated as a capability it does not provide. The remaining OOH/audio boundary is a coverage limit, not a relaxation of fields that already ship.
+- **Catalog reporting selection.** `advanced_delivery_reporting` selects a native wholesale sandbox product advertising format breakdown; `time_based_delivery_reporting` independently selects one listing `time_based_views`.
+- **Catalog reporting not applicable.** `fixture_unsatisfied`, projected `not_applicable`, means no qualifying product was found. The feature was not tested; this is neither a pass nor proof of nonsupport. Fixture discovery errors and broken selected support still fail.
+- **Catalog reporting pass scope.** A pass covers required response fields, metric narrowing and legal returned measurements. A pass does not attest that the seller returns time-based views; the controller cannot inject view observations. A one-row format request cannot prove multi-row ordering or selection from an unobserved population. Mature metric value-or-disclosure accountability remains [#8412](https://github.com/adcontextprotocol/adcp/issues/8412).
+- **Catalog reporting untested paths.** Only non-guaranteed, fixed-price CPM products are graded. Guaranteed, auction-priced, brief-only and seeded-only support is not tested, and the card cannot yet distinguish this from nonsupport ([#8416](https://github.com/adcontextprotocol/adcp/issues/8416)). Collection scenarios are unchanged. The scheduled heartbeat runs bare `comply()` without selected-price budget or authored-brand derivation until [SDK #3381](https://github.com/adcontextprotocol/adcp-client/issues/3381) and [SDK #3382](https://github.com/adcontextprotocol/adcp-client/issues/3382) are addressed. Adapter and SDK details are in the [qualification memo](https://github.com/adcontextprotocol/adcp/blob/main/specs/optional-product-reporting-qualification.md).
+- **OOH/audio journey coverage remains bounded.** Expanded OOH/audio measurement identities still have schema and focused contract-test coverage rather than one complete discovery → purchase → delivery journey per metric family.
+- **Audience activation coverage.** Audience activation has capability-gated storyboards for both discovery and execution: `audience_activation_discovery` covers the catalog-wide union, per-product declarations, vendor matching, OR/wildcard filter semantics, and exclusion of undeclared products; `external_audience_source_binding` binds a declared shared dataset, preserves its source and match state through discovery, uses the resulting audience in a media buy, and rejects undeclared rails or cross-transport updates. The reference seller deliberately advertises `dataset_query`, not `platform_distribution`, so the latter remains not applicable rather than simulated as a capability it does not provide.
 - **No automated enforcement of platform agnosticism** beyond the `check:platform-agnostic` lint scanning property names. A richer check covering schema semantics is future work.
 - **No latency or response-time SLA.** The protocol has no normative expectations for how quickly an agent must respond (unlike OpenRTB's per-auction `tmax`). Buyers and sellers negotiate timing through the commercial relationship or through task-specific [accountability terms](/docs/media-buy/advanced-topics/accountability). A structured SLA declaration is deferred.
 - **No runtime `get_schema` tool on the wire — by design in 3.x.** MCP exposes task input shapes through `tools/list.inputSchema`. The [AdCP A2A profile](/docs/building/by-layer/L0/a2a-profile-extension) advertises exact task IDs in `AgentCard.skills[]`, while SDKs and adapters validate those tasks against the selected version's bundled schemas. Coding agents can also use the [packaged SKILL.md files](https://github.com/adcontextprotocol/adcp/tree/main/skills). AdCP therefore does not add a second `get_schema` capability that returns request and response shapes from the live agent; [#3057](https://github.com/adcontextprotocol/adcp/issues/3057) was closed with that decision. The tradeoff is that the canonical response schemas and A2A validation source remain versioned artifacts rather than live task output. Reconsider the decision if private task extensions become common, a primary transport lacks any static capability descriptor, or runtime schema negotiation becomes materially different from selecting a published AdCP version.

package.json

@@ -124,7 +124,7 @@
     "test:sdk-shims": "node --test --test-force-exit --test-timeout=30000 tests/lint-sdk-shims.test.cjs tests/sdk-runtime-compat.test.cjs",
     "test:tenant-routing": "vitest run --config server/vitest.config.ts server/src/training-agent/tenants/tenant-smoke.test.ts",
     "test:format-identity-boundaries": "node --test --test-force-exit --test-timeout=30000 tests/lint-format-identity-boundaries.test.cjs",
-    "test:sdk-runner-capability-gates": "node --test --test-force-exit --test-timeout=30000 tests/sdk-runner-capability-gates.test.cjs && node --test --test-timeout=30000 tests/evaluator-allowlist-runner.test.cjs",
+    "test:sdk-runner-capability-gates": "node --test --test-force-exit --test-timeout=30000 tests/sdk-runner-capability-gates.test.cjs && node --test --test-timeout=30000 tests/evaluator-allowlist-runner.test.cjs && node --test --test-timeout=30000 tests/optional-product-reporting.test.cjs tests/optional-product-reporting-public.test.cjs tests/catalog-reporting-preflight.test.cjs tests/catalog-reporting-preflight-security.test.cjs",
     "test:callapi-state-change": "node --test --test-force-exit --test-timeout=30000 tests/lint-callapi-state-change.test.cjs",
     "test:cdn-artifacts-worker": "node --test --test-force-exit --test-timeout=30000 tests/artifact-cdn-worker.test.cjs",
     "test:sign-protocol-tarball": "node --test --test-force-exit --test-timeout=30000 tests/sign-protocol-tarball.test.cjs",

scripts/lint-storyboard-context-resolution.cjs

@@ -347,6 +347,18 @@ function lintDoc(doc, filePath) {
   // only checks key presence — but running the real substitution keeps this
   // lint honest about *which* strings the runner rewrites.
   const context = Object.assign(Object.create(null), rootContext);
+  // Declared fixture outputs are committed before authored steps. Recognize
+  // only named producers, never the entire protected namespace. The fixture
+  // declaration/schema lint separately validates their path/witness/snapshot.
+  for (const kind of ['products', 'pricing_options']) {
+    for (const declaration of doc.fixture_resolution?.[kind] ?? []) {
+      for (const output of declaration.context_outputs ?? []) {
+        if (typeof output.key === 'string' && /^__adcp_fixture_[A-Za-z0-9_]+$/.test(output.key)) {
+          context[output.key] = `<fixture:${output.key}>`;
+        }
+      }
+    }
+  }
 
   doc.phases.forEach((phase, phaseIndex) => {
     if (!phase || typeof phase !== 'object') return;

server/src/addie/config-version.ts

@@ -30,7 +30,7 @@ import { loadRules, loadResponseStyle } from './rules/index.js';
  * Format: YYYY.MM.N where N is incremented for multiple changes in a month
  * Example: 2025.01.1, 2025.01.2, 2025.02.1
  */
-export const CODE_VERSION = '2026.10.11';
+export const CODE_VERSION = '2026.10.12';
 
 // Types
 export interface ConfigVersion {

server/src/addie/mcp/member-tools.ts

@@ -14,6 +14,7 @@ import { isAuthenticatedUserAAOAdmin, type AAOAdminPrincipal } from '../admin-st
  */
 
 import { createCipheriv, createDecipheriv, randomBytes, scryptSync } from 'node:crypto';
+import { withHostedReportingBudget, runCatalogReportingStoryboard as runStoryboard, runCatalogReportingStoryboardStep as runStoryboardStep } from '../../compliance/catalog-reporting-runner.js';
 import { createLogger } from '../../logger.js';
 import { classifyProbeError, probeReasonLabel } from '../../utils/probe-error.js';
 import { validateExternalUrl } from '../../utils/url-security.js';
@@ -75,8 +76,6 @@ import {
   listAllComplianceStoryboards,
   getComplianceStoryboardById,
   resolveStoryboardsForCapabilities,
-  runStoryboard,
-  runStoryboardStep,
   createTestClient,
   testCapabilityDiscovery,
   type AgentProfile,
@@ -5792,10 +5791,11 @@ export function createMemberToolHandlers(
       // authored against; the run-auth bearer substitution no-ops when the
       // kit already carries auth.
       const declaredTestKit = getTestKitForStoryboard(storyboardId, runOptions);
-      const storyboardRunOptions = withSdkSafeTransport(withHostedStoryboardRunOptions({
+      const storyboardRunOptions = withSdkSafeTransport(withHostedReportingBudget(sb, withHostedStoryboardRunOptions({
+        signal: AbortSignal.timeout(90_000),
         ...(declaredTestKit && { test_kit: declaredTestKit }),
         ...(authOption && { auth: authOption }),
-      }, runTarget, authProbeTask));
+      }, runTarget, authProbeTask)));
       // adcp#7758 — `requires: [multi_agent]` storyboards route governance
       // steps to the public governance agent, as the fixed hosted-grader buyer
       // agent, and everything else to the agent under test, which receives a
@@ -5891,8 +5891,22 @@ export function createMemberToolHandlers(
       output += `## ${result.storyboard_title}\n\n`;
       output += `**Agent:** ${resolved.resolvedUrl}\n`;
       output += `**Compliance target:** ${formatComplianceTarget(runTarget)}\n`;
-      output += `**Result:** ${result.overall_passed ? 'PASSED' : 'FAILED'} — ${result.passed_count} passed, ${result.failed_count} failed, ${result.skipped_count} skipped, ${result.validations_advisory_failed ?? 0} advisory validation(s) failed\n`;
+      const coverageGaps = result.coverage_gaps ?? [];
+      const catalogReportingGap = ['media_buy_seller/advanced_delivery_reporting', 'media_buy_seller/time_based_delivery_reporting'].includes(storyboardId)
+        && coverageGaps.some(gap => gap.reason === 'fixture_unsatisfied');
+      const resultLabel = !result.overall_passed || result.failed_count > 0 ? 'FAILED'
+        : catalogReportingGap ? 'NOT APPLICABLE — no qualifying product in your wholesale sandbox catalog (feature not tested)'
+        : coverageGaps.length > 0 ? 'INCOMPLETE — coverage gap' : 'PASSED';
+      output += `**Result:** ${resultLabel} — ${result.passed_count} passed, ${result.failed_count} failed, ${result.skipped_count} skipped, ${result.validations_advisory_failed ?? 0} advisory validation(s) failed\n`;
       output += `**Duration:** ${(result.total_duration_ms / 1000).toFixed(1)}s\n\n`;
+      for (const gap of coverageGaps) {
+        const label = gap.reason === 'fixture_unsatisfied' ? 'Not applicable for this run' : 'Coverage gap';
+        output += `**${label}:** ${renderStoryboardDiagnostic(gap.reason, 160)} — ${renderStoryboardDiagnostic(gap.detail, RUNNER_ERROR_MAX_LEN)}\n`;
+        if (gap.reason === 'fixture_unsatisfied') {
+          output += 'No qualifying sandbox fixture was found; this feature support was not tested.\n';
+        }
+      }
+      if (coverageGaps.length > 0) output += '\n';
 
       let anyFixPlans = false;
       for (const phase of result.phases) {
@@ -6050,11 +6064,12 @@ export function createMemberToolHandlers(
         resolved.resolvedUrl,
         sb,
         resolvedStepId,
-        withSdkSafeTransport(withHostedStoryboardRunOptions({
+        withSdkSafeTransport(withHostedReportingBudget(sb, withHostedStoryboardRunOptions({
+          signal: AbortSignal.timeout(90_000),
           ...(declaredStepTestKit && { test_kit: declaredStepTestKit }),
           context,
           ...(authOption && { auth: authOption }),
-        }, runTarget, authProbeTask)),
+        }, runTarget, authProbeTask))),
       );
 
       // runStoryboardStep catches its own throws and surfaces them as