Add ts CLI ad-template config diagnostics and browser audit - #823
Add ts CLI ad-template config diagnostics and browser audit#823prk-Jr wants to merge 269 commits into
Conversation
- build_bid_map now returns serde_json::Map with full bid objects (hb_pb,
hb_bidder, hb_adid, nurl, burl) instead of a plain CPM string map
- build_bids_script / build_ad_slots_script now emit full <script> tags
using JSON.parse("…") for safe inline embedding; add html_escape_for_script helper
- build_ad_slots_script uses correct property names (gam_unit_path, div_id,
formats, targeting) matching the client-side TSJS bundle expectations
- Replace map_or(false, …) with is_some_and(…) on lines 546, 549, 567
- Add # Panics doc sections to handle_publisher_request and create_html_processor
…nities.toml at startup
… from slotRenderEnded; slim-Prebid lazy loader
- Enable APS and adserver_mock in auction config; set providers and mediator - Increase auction_timeout_ms from 500ms to 3000ms — 500ms was too tight for HTTPS round-trips to mocktioneer, leaving the mediator zero budget - Fix mediation request: send numeric price instead of opaque encoded_price; mocktioneer requires a decoded price field and does not support encoded_price - Expand creative-opportunities slot page_patterns to include /news/**
Define SlotRenderEndedEvent, SlotRenderEvent, and TestWindow types to eliminate all @typescript-eslint/no-explicit-any violations in gpt/index.ts and gpt/index.test.ts. Extend GptWindow with __tsjs_slim_prebid_url so installSlimPrebidLoader avoids the any cast.
Set gam_network_id to 88059007 (autoblog production network). Update atf_sidebar_ad slot to /88059007/autoblog/news with div_id ad-atf_sidebar-0-_r_2_ (desktop ATF sidebar, 300x250); restrict page_patterns to article paths only (/20**, /news/**) since that div does not exist on the homepage. Add homepage_header_ad slot targeting /88059007/autoblog/homepage with ad-header-0-_R_jpalubtak5lb_ for 970x90/728x90/970x250 leaderboard formats. Reduce auction_timeout_ms from 3000 to 500 to cap TTFB at the spec-recommended ceiling.
The bids script set window.__ts_bids but never invoked the __tsAdInit function, leaving GPT slots undefined and server-side targeting (hb_pb, hb_bidder) never applied. Both the winning-bid path (build_bids_script) and the no-auction fallback (html_processor None branch) now guard-call the function after the assignment.
Adds [slot.providers.pbs.bidders] support so PBS bidder params live in creative-opportunities.toml alongside APS params, without needing PBS stored requests configured server-side. PrebidAuctionProvider now sends imp.ext.prebid.storedrequest.id as a fallback for slots with no inline PBS params, and skips non-PBS provider keys (e.g. "aps") that belong to separate auction providers. PrebidImpExt gains an optional storedrequest field; empty bidder maps are omitted during serialisation. Wires mocktioneer and criteo (placeholder IDs) for both autoblog creative-opportunity slots.
…id entries from destroyed slots no longer persist
…all HTML responses per spec §4.7 + §8
… Purpose 1 when gdpr_applies
…patch_auction dispatch_auction was building AuctionContext with a placeholder Request (GET https://placeholder.invalid/) that carried no headers. Prebid's request_bids copies User-Agent, x-forwarded-for, Referer, Accept-Language, and cookies from context.request before sending to Prebid Server, so SSPs received stripped requests and returned empty bids. Fix: dispatch SSP requests before req.send_async(), using the original request directly as AuctionContext.request. DispatchedAuction holds no lifetime reference to Request, so the borrow ends at return and req can be modified (restrict_accept_encoding, Host header) and sent to origin immediately after.
In collect_dispatched_auction, the select loop checked `auction_start.elapsed() >= deadline` after each SSP response and broke early if the 1500ms budget had elapsed. When origin TTFB + body download exceeded the auction budget, the check fired after collecting the first SSP response, abandoning the second SSP's already-buffered response. This left responses with only one (possibly errored) SSP, causing remaining_ms == 0 which skipped the mediator, and select_winning_bids on the partial set returned zero bids. The deadline break is wrong in this context: SSP HTTP connections are already bounded by the backend first_byte_timeout set at dispatch time (1000ms per provider). By the time collect is called at origin EOF, all SSPs have either responded or been errored by Fastly's host. The select() calls drain instantly — no WASM-level deadline enforcement is needed or safe. Also add info-level log statements at dispatch, collect, and write_bids_to_state to make the auction pipeline observable without requiring a dashboard.
# Conflicts: # crates/trusted-server-core/src/publisher.rs
aram356
left a comment
aram356
left a comment
There was a problem hiding this comment.
Summary
Round-7 re-review at 37006d74e. The resolution commit closes all 19 round-6 findings: both blockers are fixed with committed regression tests reproducing the exact round-6 probes (ambiguous-stem staleness, registry/request volatile refusals, broad-prefix-plus-exact-sibling observation, tuned-field split), all 12 posted suggestions landed, derive_more is properly re-gated with the derive and impl Error restored, and every docs item is applied accurately — suite 431 passed / 0 failed, clippy -D warnings and fmt clean. Requesting changes for one defect the fix introduced: the observation superset now also feeds observed_literals, so a refused ambiguous stem disqualifies the configured prefix from merge routing and produces a note naming a "configured literal prefix" that does not exist — contradicting the cli.md paragraph the same commit wrote.
2 of the inline comments below carry a one-click GitHub
suggestion, both verified in a scratch worktree in isolation and together (fmt, clippy-D warnings, full CLI suite, docs prettier, byte-exact drift checks). A third candidate — the note-wording fix inside the blocking finding — is delivered as prose because the committed test atslot_toml.rs:1756pins the current wording and must change with it, which a single-range suggestion cannot do.
Blocking
🔧 wrench
- Refused stems leak into
observed_literals, changing prefix routing and mislabeling the note — see inline atcrates/trusted-server-cli/src/commands/audit/generate/slot_toml.rs:233
Non-blocking
🤔 thinking / ⛏ nitpick
- 🤔 The case-alternation gate narrows the token rule but reopens a hole for single-case random suffixes — see inline at
crates/trusted-server-cli/src/commands/audit/generate/gpt_slots.rs:294 - ⛏
matching_observed_div_indexesallocates aVecper evidence div — see inline atcrates/trusted-server-cli/src/commands/audit/generate/slot_toml.rs:418(suggestion) - ⛏ cli.md documents the split but not the split note — see inline at
docs/guide/cli.md:280(suggestion) - ⛏ Two names for one 5-second CDP bound in a single call graph — see inline at
crates/trusted-server-cli/src/commands/audit/browser_scroll.rs:8
Cross-cutting / body-level findings
- ⛏ Commit message uses a forbidden semantic prefix:
fccfcd484isfix(cli): resolve ad-template generation review gaps— CLAUDE.md's commit conventions ban semantic prefixes and every other authored commit on the branch follows the sentence-case imperative style. Reword on the next amend, or moot it by squash-merging. - 📝 Squash-merge reminder stands: HEAD is clean of real-world tokens, but the branch's 216 authored commits still carry customer names in commit messages (e.g. "Update creative-opportunities config to real autoblog.com GAM values"). A squash-merge keeps them out of main's history.
CI Status
- Analyze (actions): PASS
- Analyze (javascript-typescript): PASS (×2)
- Analyze (rust): PASS
- CodeQL: PASS
- browser integration tests: PASS
- cargo check (cloudflare native + wasm32-unknown-unknown): PASS
- cargo check/build/test (spin native + wasm32-wasip1): PASS
- cargo fmt: PASS (required)
- cargo test: PASS (required)
- cargo test (axum native): PASS
- cargo test (cross-adapter parity): PASS
- cargo test (ts CLI, native): PASS
- format-docs: PASS (required)
- format-typescript: PASS (required)
- integration tests: PASS
- integration tests (Fastly EC lifecycle): PASS
- prepare integration artifacts: PASS
- vitest: PASS
Local verification at 37006d74e: CLI 431 lib + integration suites passed, 0 failed; clippy -D warnings clean; fmt clean; docs prettier clean.
|
Round-seven body-level follow-up:
|
aram356
left a comment
aram356
left a comment
There was a problem hiding this comment.
Summary
Round-8 re-review at 0ede1a1fd. c484d7158 resolves all six round-7 findings with evidence: the observed/literals split is plumbed through a new EvidenceTable::observed_literals() and makes the A/B routing probe symmetric, the token gate now catches the four hex and single-case hashes while dropping both CamelCase false positives, and the three nitpicks (iterator return, cli.md sentence, timeout-constant unification) landed with no behavior change at any call site. Both commit messages follow the sentence-case convention. Requesting changes for an over-correction in that same token fix — ALL-CAPS placement labels including four standard IAB format names are now refused as volatile — and for three structural issues in 03743c153, which adds a GPT-stability wait that sits outside --settle-max-ms, exits on a weaker criterion than the one implemented twenty lines below it, and returns partial evidence silently.
1 inline comment carries a one-click
suggestion, verified in a scratch worktree:cargo fmt --all -- --checkclean,cargo clippy -p trusted-server-cli --all-targets -- -D warningsclean, full CLI suite green at 435 passed / 0 failed — with the one companion fixture edit named in that comment, which a single-range suggestion cannot carry. The remaining comments describe fixes in prose because they span files, need a new test seam, or are design calls.
Blocking
🔧 wrench
- ALL-CAPS placement labels refused as volatile families — see inline at
crates/trusted-server-cli/src/commands/audit/generate/gpt_slots.rs:309(suggestion) - GPT-stability wait escapes
--settle-max-ms, up to 5s per page per profile — see inline atcrates/trusted-server-cli/src/commands/audit/generate/browser_collector.rs:966 - Stability criterion exits on two samples 250ms apart, weaker than the sibling idiom — see inline at
crates/trusted-server-cli/src/commands/audit/generate/browser_collector.rs:999 - Bound expiry returns partial evidence silently; timeout warning misattributes the cause — see inline at
crates/trusted-server-cli/src/commands/audit/generate/browser_collector.rs:976
❓ question
- Should
verify/pageget the same wait, or is the asymmetry structural? — see inline atcrates/trusted-server-cli/src/commands/audit/generate/browser_collector.rs:966
Non-blocking
♻️ refactor / 🤔 thinking / ⛏ nitpick
- ♻️ New behavior has zero deterministic coverage (all three fixtures are Chrome-gated) — see inline at
crates/trusted-server-cli/src/commands/audit/generate/browser_collector.rs:1435 - 🤔 The new fixture cannot detect the early-exit weakness — see inline at
crates/trusted-server-cli/src/commands/audit/generate/browser_collector.rs:1195 - ⛏ Order-sensitivity undocumented; the wait reaches no docs — see inline at
crates/trusted-server-cli/src/commands/audit/generate/browser_collector.rs:962 - ⛏
distinct_ascii_bytesallocates 256 bytes for a ≤64-byte question — see inline atcrates/trusted-server-cli/src/commands/audit/generate/gpt_slots.rs:321 - ⛏ Two assertions without messages in the multi-ancestor split test — see inline at
crates/trusted-server-cli/src/commands/audit/generate/slot_toml.rs:1838
Cross-cutting / body-level findings
- 📝 No progress output during the stability wait:
CollectionProgressemits nothing whilecollect_stable_gpt_slotspolls, so a page that burns the full budget shows as a frozenAuditing … [n/m]: /pathline for 5s with no explanation. Worth a phase line or a note once the budget question above is settled. - 📝 Squash-merge reminder stands: HEAD remains clean of real-world tokens, but the branch's authored commit messages still carry customer names from earlier rounds. A squash-merge keeps them out of main's history.
CI Status
- Analyze (actions): PASS
- Analyze (javascript-typescript): PASS (×2)
- Analyze (rust): PASS
- CodeQL: PASS
- browser integration tests: PASS
- cargo check (cloudflare native + wasm32-unknown-unknown): PASS
- cargo check/build/test (spin native + wasm32-wasip1): PASS
- cargo fmt: PASS (required)
- cargo test: PASS (required)
- cargo test (axum native): PASS
- cargo test (cross-adapter parity): PASS
- cargo test (ts CLI, native): PASS
- format-docs: PASS (required)
- format-typescript: PASS (required)
- integration tests: PASS
- integration tests (Fastly EC lifecycle): PASS
- prepare integration artifacts: PASS
- vitest: PASS
Local verification at 0ede1a1fd: CLI 436 lib tests + integration suites passed, 0 failed; clippy -D warnings clean; fmt clean.
The single-case branch of `looks_random_suffix` returned before the word-shape check ever ran, so any all-caps run of four or more distinct letters read as a hash. Four standard IAB format names were among the casualties: a div id such as `promo-20260820BILLBOARD-sidebar` had its whole family dropped from the generated config, with a warning telling the operator to change markup that is in fact stable. The mirror case was missing entirely, since a lowercase hash like `zzqxwvkm` passed. Make the branch case-symmetric and gate it on vowel density instead of case, keeping the pure-hex check ahead of it so `deadbeef` and `ABCD1234` still catch. Swap the `ABCDEFGH` fixture for `XKMPQRST`: a sequential alphabet run carries a word's vowel spread and is no longer a hash shape. Also replace the 256-byte flag array in `distinct_ascii_bytes` with a four-word bitset, and give the two message-less assertions in the multi-ancestor split test the wording the third one already had.
The wait budgeted against the hardcoded five-second page-operation constant and started its own clock after both settle passes had already finished, so worst-case page time grew by an amount no flag could tune. Take the budget from `settle_max` and the dwell from `settle_quiet` instead, and give each CDP read back its own fixed bound, so a wedged evaluate is reported distinctly from an exhausted budget. Exit on a dwell rather than on one matching pair. Two reads 250ms apart can both observe the same registration burst and miss the next, which is the batching the wait exists to survive; mirror the `quiet_since` idiom of `wait_for_page_settle` and require the reading to repeat for the dwell window. Warn when the budget expires with a snapshot still in flight. Both expiry paths returned silently while every other exit warned, so the one condition the wait exists to detect produced a partial slot list indistinguishable from a stabilized read. An empty registry stays silent, because the caller's googletag-state diagnostic names the actual cause. Extract the classification into `gpt_registry_reading` so the empty-reset, order-sensitivity and repeat-match rules have deterministic coverage off the Chrome-gated path, and document why order sensitivity is kept: slot discovery takes the network id from the first usable entry and emits slots in registry order. Add a fixture whose second burst lands on a real timer rather than on observation, which the previous criterion cannot pass, and a note at the call site recording why verification needs no counterpart wait.
Three conflicts, all from main's EdgeZero v0.0.7 upgrade (0f8b44d). `run.rs` needed a union rather than a side. Main added the `active-version`, `healthcheck`, `rollback` and `config gc` commands against a `dispatch` returning `Result<(), String>`, while this branch changed that signature to `Result<RunOutcome, String>` to carry assertion exit codes. Main's four new arms therefore take the branch's `.map(|()| RunOutcome::Success)` wrapper. Two of those arms had auto-merged outside the conflict markers without it, which would not have compiled. Kept both test sets, placed at main's own insertion points so the merge does not reorder the tests main already had. `parses_audit_with_default_outputs` and `parses_audit_with_custom_outputs` stay deleted: they cover the merge-base audit surface this branch replaced, and main never touched them. `Cargo.lock` needed the `toml_edit` entry version-qualified, because main's upgrade puts a second `toml_edit` in the graph; the workspace pins 0.23.10, so the CLI keeps that one alongside this branch's `tracing`. `docs/guide/cli.md` had both sides adding a section at the same anchor. Main's `config gc` prose continues the `config push` discussion above it, so it leads, followed by this branch's ad-template diagnostics heading.
aram356
left a comment
aram356
left a comment
There was a problem hiding this comment.
Summary
Round-8 review at 8f0715c59. The two newest commits independently resolved four of this round's concerns before they were posted: the GPT stability wait is now bounded by the operator's --settle-max-ms with the per-read CDP bound kept separate, the ALL-CAPS placement-label false positives are fixed by a case-symmetric vowel gate (probed in both directions: LEADERBOARD-class labels eligible, XKMPQRST/deadbeef/ABCD1234 still refused), the extracted registry-reading helpers gained deterministic tests, and the wait is documented in cli.md. The earlier cold-start Chrome flake did not recur across five runs. Requesting changes for the two defects that survived: the empty-registry path still burns the full budget — now 12s instead of 5s per GPT-less page (measured 19.2s end-to-end on a googletag: undefined page) — and the refused-plus-written stem gap still lets a configured prefix swallow a sibling with an inherited floor price (reproduced through the real merge path), plus one unanswered direction question on the multi-ancestor split warnings.
1 of the inline comments below carries a one-click GitHub
suggestion, verified in a scratch worktree (fmt, clippy-D warnings, full CLI suite, byte-exact drift check). The two blocking fixes are delivered in prose: the empty-registry exit needs loop-state changes across three regions, and theobserved_literalsfilter change must land together with the test atevidence.rs:462-483that codifies the current behavior.
Blocking
🔧 wrench
- Empty-registry burn survived the re-bounding and grew 5s → 12s per GPT-less page — see inline at
crates/trusted-server-cli/src/commands/audit/generate/browser_collector.rs:1060 - A refused-plus-written stem still drops out of
observed_literals, re-arming the configured prefix — see inline atcrates/trusted-server-cli/src/commands/audit/generate/evidence.rs:215
❓ question
- Split warnings name every tuned ancestor while routing is longest-wins — which direction? — see inline at
crates/trusted-server-cli/src/commands/audit/generate/slot_toml.rs:294
Non-blocking
🤔 thinking / ⛏ nitpick
- 🤔 Worst-case page time is now 3×
--settle-max-ms, documented as design — see inline atcrates/trusted-server-cli/src/commands/audit/generate/browser_collector.rs:711 - 🤔 The vowel gate still refuses vowel-free consonant-abbreviation labels — see inline at
crates/trusted-server-cli/src/commands/audit/generate/gpt_slots.rs:317 - ⛏
--settle-max-ms 0silently skips the GPT read — see inline atcrates/trusted-server-cli/src/commands/audit/generate/browser_collector.rs:1031 - ⛏ The "do not port this into
audit::browser" comment overclaims its scope — see inline atcrates/trusted-server-cli/src/commands/audit/generate/browser_collector.rs:705(suggestion) - ⛏ Third name for the same 5-second CDP bound persists — see inline at
crates/trusted-server-cli/src/commands/audit/generate/browser_collector.rs:37
CI Status
- Analyze (actions): PASS
- Analyze (javascript-typescript): PASS
- Analyze (javascript-typescript): PASS
- Analyze (rust): PASS
- CodeQL: PASS
- browser integration tests: PASS
- cargo check (cloudflare native + wasm32-unknown-unknown): PASS
- cargo check/build/test (spin native + wasm32-wasip1): PASS
- cargo fmt: PASS (required)
- cargo test: PASS (required)
- cargo test (axum native): PASS
- cargo test (cross-adapter parity): PASS
- cargo test (ts CLI, native): PASS
- format-docs: PASS (required)
- format-typescript: PASS (required)
- integration tests: PASS
- integration tests (Fastly EC lifecycle): PASS
- prepare integration artifacts: PASS
- vitest: PASS
Local verification at 8f0715c59: CLI suites 457 lib + 35 integration tests passed, 0 failed; Chrome-gated generate set 4 passed under TS_AUDIT_BROWSER_TESTS=1; clippy -D warnings clean; fmt clean.
Preserve JavaScript asset proxy candidate generation in the relocated audit generator while retaining ad-template slot discovery.
Chrome opens several sockets per navigation: the document request, socket pool preconnects that close without sending anything, and speculative /favicon.ico, /robots.txt and /sitemap.xml fetches. The GPT fixture served exactly one accepted connection and panicked on a connection that carried no request, so whenever a preconnect won the accept race the listener was gone before the navigation landed and the collector failed with net::ERR_CONNECTION_REFUSED. Replace the one-shot fixture with a server that accepts until the returned guard is dropped, answers each connection on its own thread so a silent socket cannot stall the document request, and treats a request-less socket as normal. Cover both failure modes with tests that need no browser.
aram356
left a comment
aram356
left a comment
There was a problem hiding this comment.
Summary
Review at c18e59813. Five of the eight findings carried into this pass are resolved with tests that pin the new behavior: the GPT-registry poll now exits on a repeated empty read (re-measured, not inferred — the lazy-slot Chrome test dropped from 18.8s to 6.5s), observed_literals keeps a stem that was refused on one page but written as a concrete slot from another, split warnings follow longest-wins routing, a zero settle budget still takes one snapshot, and the duplicated 5-second CDP constant is down to one definition. The observed_literals fix is worth singling out for how it was done: the test that encoded the old behavior was renamed and re-pointed to assert the written-slot overlap it had never covered, and the end-to-end merge probe now lives in-tree and runs in both refusal orders.
Requesting changes for one issue this round introduced: the browser-fixture server answers Chrome's speculative requests with the full HTML document, which I bisected to a ~25% flake in the Chrome-gated tests.
The single inline
suggestionbelow was verified in a scratch worktree: applied, thencargo fmt --all -- --check,cargo clippy -p trusted-server-cli --all-targets --all-features -- -D warnings, and the full 509-test CLI suite, all clean with a byte-exact drift check — plus four Chrome-gated runs of the affected test (3.2–4.0s, 4/4 pass) on top of the six the analysis pass ran.
Blocking
🔧 wrench
- Fixture server answers speculative browser requests with the full document, flaking the Chrome-gated tests — see inline at
crates/trusted-server-cli/src/commands/audit/generate/browser_collector.rs:1496(suggestion)
Non-blocking
🤔 thinking / ⛏ nitpick
- 🤔 Worst-case page time is still 3×
--settle-max-ms— see inline atcrates/trusted-server-cli/src/commands/audit/generate/browser_collector.rs:712 - 🤔 The two-empty-read exit is a magic constant with no escape, and
--settle-max-msno longer covers the GPT-less path — see inline atcrates/trusted-server-cli/src/commands/audit/generate/browser_collector.rs:1080 - 🤔 Vowel-gate residue closed by concession rather than fix — see inline at
crates/trusted-server-cli/src/commands/audit/generate/gpt_slots.rs:316 - ⛏ Fixture acceptor polls on a 5ms sleep instead of blocking — see inline at
crates/trusted-server-cli/src/commands/audit/generate/browser_collector.rs:1441
Cross-cutting / body-level findings
- 📝 The tokio gating change is correct and needs no action. Moving
tokio = { features = ["test-util"] }fromcfg(target_os = "macos")tocfg(not(target_arch = "wasm32"))dev-dependencies is necessary for the six new#[tokio::test(start_paused = true)]tests, which are not macOS-gated. As a dev-dependency it cannot reach a production build, and the CLI is only ever built for host triples with an explicit--target, so the predicate excludes nothing in practice. No feature-unification risk into the wasm adapters. - 📝 Squash-merge reminder. HEAD remains clean of real-world identifiers, but the branch's authored commits still carry customer names in their commit messages. A squash keeps them out of
main's history.
CI Status
All 19 checks PASS at c18e59813 (Analyze actions / javascript-typescript ×2 / rust, CodeQL, browser integration tests, cargo check cloudflare + spin, cargo fmt (required), cargo test (required), cargo test axum / cross-adapter parity / ts CLI native, format-docs (required), format-typescript (required), integration tests, integration tests Fastly EC lifecycle, prepare integration artifacts, vitest).
Local verification at c18e59813: 509 tests passing across the CLI lib and integration targets, 13 ignored, 0 failed; clippy -D warnings clean; cargo fmt --all -- --check clean; Chrome-gated set passes as a batch but is flaky standalone at HEAD — see the blocking finding.
Return empty 404 responses for speculative fixture requests and wake the blocking acceptor on shutdown. Pin routing with real speculative paths. Share the generation settle budget across initial, post-scroll, and GPT waits while retaining a snapshot after exhaustion. Document the shared budget and the two-empty-poll shortcut, with Chrome regression coverage.
3 participants
Summary
[creative_opportunities]) configuration: static path/slot diagnostics viats config ad-templates …, and browser-backed live verification viats audit …(local Chrome/Chromium over CDP).ts audit ad-templates generate <url>to bootstrap[creative_opportunities]from a live site. One run crawls the publisher's sections (sitemap viarobots.txt, else navigation links), samples a landing page and an article per section, reconciles each slot across the pages it appeared on, and writes the result into an existingtrusted-server.tomlin place, preserving every other section and comment.{network_id}/{section}ad-unit template plus thesection_root/section_segmentpolicy it depends on, instead of pinning each slot to the one literal path it happened to be scraped from. A wrong template makes a publisher bid against inventory that does not exist, so inference refuses rather than guesses — see the table below.Settings::from_toml, the same load path the runtime uses at startup, on the--dry-runpath too. An unloadabletrusted-server.tomlis a full-site outage once pushed, not a degraded ad stack.chromiumoxide) are excluded from thewasm32-wasip1build, and the runtime ad-stack gate is shared withpublisher.rsso the CLI cannot drift from server behavior.closes #701
Registry and slot matching
Generation stops its final GPT wait after two consecutive empty snapshots and takes one snapshot even with a zero settle budget. A nonempty registry still needs the configured dwell window; late bursts reset that window. The documented per-phase budgets remain independent so slow page settling does not consume the registry stabilization budget.
A concrete slot written from one page remains literal evidence when another page refuses its stem, keeping sibling slots separate and preventing unintended floor-price inheritance. Split warnings identify only the longest matching configured ancestor. Consonant-only placement abbreviations remain a documented limitation of the random-suffix heuristic.
Artifact generation also retains disabled JavaScript asset-proxy candidates and their summary counts alongside discovered ad-template slots.
Changes
trusted-server-core/src/creative_opportunities.rs[creative_opportunities]config types,match_slots, sharedevaluate_ad_stack_gate;compile_page_patternas the single glob definition;derive_sectionmade public so tooling checks inference against the runtime's own derivation rather than a second implementationtrusted-server-core/src/publisher.rsshould_run_server_side_ad_stackthrough the shared gate (behavior-preserving)trusted-server-cli/src/commands/config/ad_templates.rsts config ad-templates {lint,match,check,explain}static diagnosticstrusted-server-cli/src/app_config.rstrusted-server-cli/src/ad_templates/{expected,compare,output}.rstrusted-server-cli/src/commands/audit/{mod,page,collector,browser,ad_templates}.rs,commands/audit/ad_template_collector.jsts audit page+ts audit ad-templates verify: chromiumoxide collector, read-only GPT/APS/DOM init script, verifier orchestration, cross-origin refusaltrusted-server-cli/src/commands/audit/generate/crawl_plan.rstrusted-server-cli/src/commands/audit/generate/evidence.rstrusted-server-cli/src/commands/audit/generate/unit_template.rs{network_id}/{section}inference with positional network binding, a single-varying-segment rule, the witness rule, and replay through the runtime's own renderertrusted-server-cli/src/commands/audit/generate/page_patterns.rs/newsand/news/*) without extrapolating past a witnessed sectiontrusted-server-cli/src/commands/audit/generate/validate.rsSettings::from_tomlbefore it replaces the file; a pre-existing failure downgrades to a warning so an already-broken config can still be updatedtrusted-server-cli/src/commands/audit/generate/{mod,gpt_slots}.rs_R_/_r_ids,-container, hex UUIDs)trusted-server-cli/src/commands/audit/generate/{browser_collector,collector,analyzer}.rstrusted-server-cli/src/run.rs,src/lib.rsauditnamespacetrusted-server-cli/Cargo.tomledgezero-core+serde_jsondeps (cfg-gated off wasm, like the existing browser deps)docs/guide/cli.mdts audit ad-templates generatedocumented: crawl behavior, refusal table, consent platforms, proxy auditing, the deploy-ordering contract, and the settle window as a per-phase bound rather than a whole-page onedocs/superpowers/{specs,plans}/2026-06-26-server-side-ad-template-cli*Test plan
Per CLAUDE.md, a bare
cargo test/cargo clippy --workspacefails at the workspace root — the repo has multiple wasm runtimes with runtime-specific SDKs, so the target-matched aliases are the real gate.cargo fmt --all -- --checkcargo clippy-fastly && cargo clippy-axum && cargo clippy-cloudflare && cargo clippy-cloudflare-wasm && cargo clippy-spin-native && cargo clippy-spin-wasmcargo clippy -p trusted-server-cli --target <host-triple> --all-targets --all-features -- -D warningscargo test-fastly && cargo test-axum && cargo test-cloudflare && cargo test-spincargo test --manifest-path crates/trusted-server-integration-tests/Cargo.toml --test parity(13 passed)cargo test -p trusted-server-cli --target <host-triple>cd crates/trusted-server-js/lib && npx vitest runcd crates/trusted-server-js/lib && npm run formatcd docs && npm run formatcargo build --package trusted-server-adapter-fastly --release --target wasm32-wasip1(pluscargo build -p trusted-server-cli --target wasm32-wasip1— browser deps stay out of wasm)./scripts/test-cli.sh) for evidence collection, scroll-phase attribution, and GPT registry stability — including a fixture whose second registration burst lands on a real timerts dev proxy— template and section policy inferred, per-render div-id fragments refused, generated config loads throughSettings::from_tomlNotable fixture-based coverage, all offline: crawl planning (cross-origin rejection on links and sitemap entries, utility/asset filtering, query/fragment collapsing, budget truncation), evidence reconciliation (format union, network-id conflict, fragment detection with a co-occurrence false-positive guard), and one test per template-inference refusal case.
How to use
Configure slots
In your (gitignored)
trusted-server.toml— fictional values shown:Generate slots from a live site (needs local Chrome/Chromium)
Re-running merges: a slot seen again keeps its hand-tuned fields and gains this run's patterns and newly observed formats, and a hand-written
gam_unit_pathtemplate is preserved.--replacediscards existing slots, including any template written by hand.Consent platforms. Publishers gate slot definition behind their consent platform, and the audit runs in a throwaway profile with no consent cookie — so such a site would define no slots at all and look identical to a site with no ad stack. The crawl therefore answers the two IAB interfaces every compliant platform exposes (TCF v2 and US Privacy) as a consenting, out-of-scope reader, before any page script runs. This changes only what the audit browser sees.
--no-assume-consentobserves the un-consented page instead.Auditing a production hostname served locally.
ts dev proxyserves a production hostname from a local Trusted Server; auditing through it keeps the page's origin, cookie scope, and any origin checks in the ad stack matching production rather thanlocalhost:Note that a local Trusted Server injects its own configured slots, so a run through the proxy can rediscover config it already has; slot ids absent from the current config are the publisher's own.
When generation keeps literal paths, and when it refuses
section_rootis unknownStatic diagnostics (no browser)
Browser-backed audit (needs local Chrome/Chromium)
The settle window bounds each wait phase rather than the whole page: the initial settle, the optional post-scroll settle, and — in
generateonly — a final wait for the GPT slot registry to stop changing before it is read. GPT can finish registering after the document and resource stream have gone quiet, so a single read there would sometimes miss a later batch. Raise--settle-max-msif a run warns that slot registration did not hold still.verifyneeds no such wait: its collector is injected ahead of publisher scripts and records each slot as it is defined.Shared config flags (all of the above)
Exit behavior
verifyis auditor-assist: exits0even with missing/partial evidence.--strictexits 1 when a confirmable matched slot is missing or partially confirmed; video, native, and out-of-page slots areunconfirmableand do not fail the gate. A page-level navigation failure, or a redirect that leaves the requested origin, also exits non-zero.[auction].enabled = false) mark a page "skipped" so--strictdoes not fail it.Local live test (deterministic, no external site)
Many large ad publishers block headless/non-evasive browsers, so
verifyagainst them sees a challenge page rather than the article (this tool does not evade bot detection —--cookieforwards a clearance a human already earned, and--headfulruns a visible browser). When a page comes back without slots, the run now reports GPT's observable state — whether the library reachedapiReady, how many queued commands never drained, how many scripts ran — which distinguishes "the library never loaded" from "this page has no ads".To exercise the full pipeline reliably without an external site, serve a local fixture:
For a realistic end-to-end generation run,
ts dev proxyin front of a local Trusted Server is the reliable path — see the proxy example above.Checklist
unwrap()in production code — useexpect("should ...")println!/eprintln!in library code (CLI output useswriteln!; errors uselog)