Skip to content

docs(drift): make DRIFT.md and the ElevenLabs drift legs match what runs - #465

Merged
jpr5 merged 6 commits into
mainfrom
docs/s4-drift
Sep 16, 2026
Merged

docs(drift): make DRIFT.md and the ElevenLabs drift legs match what runs#465
jpr5 merged 6 commits into
mainfrom
docs/s4-drift

Conversation

@jpr5

@jpr5 jpr5 commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

Makes DRIFT.md describe the drift suite that actually runs, and makes the two ElevenLabs drift files honest about what they grade. Found during the review pass over the contributor PRs. Docs plus drift test files only; no behaviour change to the mock, no version bump.

Because this touches src/__tests__/drift/**, the drift-live-pr job runs on this PR and spends real API credits, as the corrected CI section now states.

DRIFT.md

Running the tests

  • The environment-variable list names all twelve variables the drift legs read and which legs each one gates; a missing key skips that leg.
  • STRICT_DRIFT=1 is read by nothing in the repo and is no longer documented; legs filter on severity === "critical".
  • The partial provider/builder map is replaced by a pointer to SURFACE_REGISTRY, which has 28 entries.
  • "Adding a New Provider" includes the required registry entry with liveCoverage; the allowlist step stays because ALLOWLISTED_PATHS still exists.

Coverage tables

  • Counts come from the registry: 28 surfaces, 18 live, 10 offline.
  • Six rows that said "Covered" for routes no drift case requests (images/edits, audio/translations, ollama embed, cohere embed, stream_options.include_usage, rate-limit headers) are marked honestly with a footnote.
  • Rows added for /v1/music, /v1/music/stream, /v1/music/plan and /v1/sound-generation.
  • WebSocket test count is 11 and Interactions is 5, counted from the files.
  • Gemini Live row matches the registry (live); Realtime rows name the probed model gpt-realtime-mini, one GA family, no Beta; the realtime probe opens two GA connections per run.
  • The ElevenLabs "no key reachable" sentence states what the workflows do: CI injects ELEVENLABS_API_KEY.
  • HTTP tables sit under an HTTP heading rather than the WebSocket one.

CI schedule and cost

  • Each workflow and job is listed with its trigger and whether it spends credits, including the pull_request path filter, the drift-live-pr job, and the fix-drift 6:10 UTC cron.
  • The AG-UI schema drift lane is described.
  • Cost numbers are derived from the registry; an unverifiable weekly dollar figure is removed.

Drift test files

  • elevenlabs.drift.ts: the /v1/music/plan case is declared for what it is, passthrough conformance against a test-authored fixture with no vendor observation; the two 400-envelope cases no longer claim a vendor envelope in their titles; the house error envelope is named as ours.
  • elevenlabs.drift.ts: the live /v1/sound-generation case emits an elevenlabs drift block graded against the vendor response, which is what earns the surface its live status in the registry.
  • elevenlabs-voice.drift.ts: comments state that the key is provided in CI and that these routes have no vendor leg; the live leg keeps the vendor error body; server construction sits inside the try; afterAll no longer masks a beforeAll failure.
  • surface-registry.ts: ElevenLabs comments and coverageNote corrected to the same facts.

Checked and left alone

  • Gemini Live grading on the AUDIO modality is correct as written.
  • The voice legs were not marked live: no ElevenLabs key was available locally to run them, and the routes have no vendor leg.

Verification

Every changed DRIFT.md claim was recounted from the registry, the drift files and the two workflows (44 claims, one found false and corrected). The test changes had a three-reviewer panel; its findings were fixed or, where pre-existing, recorded as follow-ups. Formatter, lint, both typecheck configs, full test suite, build and commitlint are clean.

jpr5 added 6 commits September 16, 2026 11:58
@jpr5
docs(drift): correct the env-var list, strictness flag, provider map …
2eed9a7 
…and new-provider steps

[19] Env vars: the list named 4 keys while the drift legs read 12. Every
process.env.* read under src/__tests__/drift/ is now listed with the legs it
gates. Sources: openai-chat.drift.ts:34,125; openai-responses.drift.ts:45,129;
openai-embeddings.drift.ts:19,33; transcription.drift.ts:58,217;
ws-responses.drift.ts:25,61; ws-realtime.drift.ts:40-45,59,116,194
(OPENAI_REALTIME_KEY ?? OPENAI_API_KEY); models.drift.ts:457,471,488;
anthropic.drift.ts:35,117,772; gemini.drift.ts:28,42,645;
gemini-interactions.drift.ts:31,45; ws-gemini-live.drift.ts:102,232;
cohere.drift.ts:31,284; rerank.drift.ts:21,104; openrouter-chat.drift.ts:41,484;
openrouter-video.drift.ts:35,253; fal-queue.drift.ts:25,226;
elevenlabs.drift.ts:38,205; byteplus-video.drift.ts:64,287 (keyless canary at
:181 has no gate); providers.ts:912-916 (ARK_BASE_URL default);
ollama.drift.ts:23,32,123 (OLLAMA_MODEL default llama3.2). Every gate is
describe.skipIf/it.skipIf, so a missing key skips, never fails. "requires all
three API keys" removed as false.

[39a] STRICT_DRIFT: grep over src/ and .github/ finds no reader (only DRIFT.md
itself). Warnings never fail: legs filter severity === "critical"
(openai-chat.drift.ts:165 et al.). The strict-mode example and the "(unless
STRICT_DRIFT=1)" clause are removed; no other variable drives strictness.

[39b] Builder map listed 13 entries against 28 SURFACE_REGISTRY keys
(surface-registry.ts:92-395). Replaced with one sentence pointing at the
registry's builderFile/builderFunctions; the `Surface:` marker line is emitted
by formatDriftReport (schema.ts:484).

[20] Adding a New Provider gained the registry step: formatDriftReport throws
on an unregistered slug (schema.ts:470-474); liveCoverage has no default and
"none" requires coverageNote (surface-registry.ts:75-80);
drift-collector.test.ts:3306-3320 fails on an emitted slug missing from the
registry. The schema.ts allowlist step is kept — ALLOWLISTED_PATHS still
exists (schema.ts:196).
@jpr5
docs(drift): make the coverage tables match the registry and the drif…
4cc119b 
…t cases

Every statement below is checked against the code in this tree (f5b9b9d).

- :125 heading "WebSocket Drift Coverage" -> "Additional Drift Coverage"; the
  HTTP table and footnotes 2-5 sit under it. A "### WebSocket Protocols"
  heading is inserted above the WS table so it no longer sits under the Gemini
  Interactions heading. No content moved.
- :127 "23 core / 20 HTTP / 3 model" replaced with the registry counts:
  28 surfaces, 18 live, 10 none (src/__tests__/drift/surface-registry.ts,
  `liveCoverage:` occurrences excluding the type union).
- Rows /v1/images/edits, /v1/audio/translations, /api/embed(+/embeddings),
  /v2/embed, stream_options.include_usage, x-ratelimit/Retry-After: "Covered"
  -> "None(6)" with a new footnote. grep -rn over src/__tests__/drift/ finds no
  request for any of those paths or headers; include_usage appears only in
  openrouter-chat.drift.ts:350 (mock-only OpenRouter case). images.drift.ts
  drives /v1/images/generations only; registry `images` is liveCoverage none.
- embedContent row -> "Covered (live canary)": gemini.drift.ts:645-668 fetches
  the real endpoint but only logs keys and asserts expect(true).
- Added ElevenLabs rows: /v1/sound-generation Covered (elevenlabs.drift.ts:
  205-216 fetches api.elevenlabs.io when ELEVENLABS_API_KEY is set);
  /v1/music, /v1/music/stream, /v1/music/plan Offline (mock only)
  (elevenlabs.drift.ts:225-300 drive only instance.url).
- Footnote 4 "No ElevenLabs key is reachable from this repo" -> the workflows
  do inject ELEVENLABS_API_KEY (.github/workflows/test-drift.yml:290,:503;
  fix-drift.yml:209,:447) but elevenlabs-voice.drift.ts never reads it.
- :180 "6 verified + 2 canary = 8 WS tests" -> 11 it() blocks:
  ws-responses.drift.ts (:64,:137), ws-realtime.drift.ts (:66 canary, :116,
  :194), ws-gemini-live.drift.ts (:244,:339 live; :476,:487,:515,:549 offline
  unit) = 6 live comparisons + 1 canary + 4 unit.
- :184 "4 drift tests" -> 5; gemini-interactions.drift.ts:48,:92,:136,:182,:242.
  Added the missing "(Step[] input)" bullet.
- Gemini Live row "Unverified" -> "Verified", Tool Call check, Text "— (AUDIO
  check)": registry gemini-live liveCoverage "live"; ws-gemini-live.drift.ts
  :232-339 drives the real endpoint.
- Models line: Realtime GA probe uses gpt-realtime-mini (ws-providers.ts:704,
  :721), not gpt-realtime-2; dropped the "(was ...)" clause.
- Canary family list now matches `gaRealtimeModels` (voice-models.ts:23-30):
  adds gpt-realtime-2.1 and gpt-realtime-2.1-mini.
- Gemini Live section: "every model exposing bidiGenerateContent is a
  native-audio model" is false — gemini-3.5-transcribe-live declares it and
  emits TEXT only (ws-gemini-live.drift.ts:22-31); model selection now states
  the driveGeminiLiveAudio walk with 1007 advancing (:53-57). Graded modality
  stays AUDIO (ws-providers.ts:808 GEMINI_LIVE_RESPONSE_MODALITIES=["AUDIO"]).

Not touched (outside lines 125-231): DRIFT.md:275 still says "(one GA, one
Beta)" and "gpt-realtime-2" — the cost paragraph is a sibling's range.
@jpr5
docs(drift): state when the drift workflows actually run and what the…
df0b6a3 
…y cost

DRIFT.md "CI Schedule" claimed the drift suite runs "NOT on PR or push".
False: test-drift.yml has a `pull_request` trigger with a `paths:` filter
(.github/workflows/test-drift.yml:5-27) and a `drift-live-pr` job gated
`github.event_name == 'pull_request'` (:492-493) that runs the live
collector on head and, unless a same-UTC-day main report is reusable, on
base too (:632-660, :745-762), spending real credits per PR push. The
credit-free `agui-schema-drift` job (:34-66, no `if:`, clones ag-ui) was
never mentioned; `drift` (:67-68) and `notify` (:340-342) are the
non-PR jobs. fix-drift.yml's 6:10 UTC cron (:14-15) and its `sync` job
gate (:37-40) were absent; its re-collect gate runs the full live
collector (:56, :156-157, :203).

"Cost" replaced a stale per-run call count (23 core / 20 HTTP / ~31
calls) with the registry-derived live surface count: 18 of 28 entries in
src/__tests__/drift/surface-registry.ts carry `liveCoverage: "live"`,
10 `"none"`. Realtime is one GA connection, not "(one GA, one Beta)":
src/__tests__/drift/ws-providers.ts:692-704 (Beta retired, model
`gpt-realtime-mini`, not `gpt-realtime-2`). Dropped the unverifiable
"$0.25/week" figure.
@jpr5
test(drift): grade /v1/music/plan against the SDK MusicPrompt.Raw sha…
9da4f8b 
…pe, relabel house errors

The `/v1/music/plan` case graded aimock against a shape copied from its own
fixture input, so it could only fail on a passthrough bug while the surface is
registered `liveCoverage: "live"`. It now asserts the vendor shape the official
client parses for that route, read from the published `@elevenlabs/elevenlabs-js@2.68.0`
tarball (not a repo dependency):
  - api/resources/music/resources/compositionPlan/client/Client.js joins
    "v1/music/plan" and parses with
    serializers.music.CompositionPlanCreateResponse.parseOrThrow;
  - serialization/.../CompositionPlanCreateResponse.d.ts:
    type Raw = MusicPrompt.Raw | CompositionPlan.Raw;
  - serialization/types/MusicPrompt.d.ts Raw: { positive_global_styles: string[];
    negative_global_styles: string[]; sections: SongSection.Raw[] };
  - serialization/types/SongSection.d.ts Raw: { section_name; positive_local_styles;
    negative_local_styles; duration_ms; lines; source_from? }.
The plan handler writes fixture `content` verbatim, so the fixture is now a
MusicPrompt.Raw too. This is a secondary source, stated as such in the comment;
no live /v1/music/plan request has been made and the surface's "live" still
comes from /v1/sound-generation alone.

RED: with the SDK shape and the old fixture the case failed with 3 critical
diffs (positive_global_styles/negative_global_styles absent, sections[] string
vs object). GREEN: 7 passed, 1 skipped offline.

Hygiene from the R2-R5 reviews, same file:
  - `elevenLabsErrorShape` -> `aimockErrorEnvelopeShape`; the two 400 cases are
    named and documented as pinning aimock's house envelope (the vendor answers
    422 `{ detail: [...] }`), not ElevenLabs'.
  - the live /v1/sound-generation leg keeps the vendor error body and prints it
    in the status assertion instead of discarding it.
  - `afterAll` returns when `beforeAll` never set `instance`. Patch dance with a
    forced `beforeAll` throw: before, the report showed the real error plus
    "TypeError: Cannot read properties of undefined (reading 'server')"; after,
    only "forced beforeAll failure".
  - "/v1/music song-id header absent on plan endpoint" renamed to the route it
    hits; the plan content-type check uses toContain like its siblings.

Live run: SKIPPED. `internal-skills secret-cache resolve "elevenlabs api key"`
exited 4 (no match in 1Password), so the key was not hunted for.
@jpr5
fix(drift): tell the truth about the elevenlabs-voice key and harden …
e4420d7 
…its lifecycle

The `elevenlabs-voice.drift.ts` header and the `elevenlabs-voice` registry note
said no ElevenLabs key is reachable from this repo. `.github/workflows/test-drift.yml`
(:290, :503) and `fix-drift.yml` (:209, :447) pass `secrets.ELEVENLABS_API_KEY`
into the drift run. The comments now say the key is wired into CI and that the
only reason the surface is offline is that no case in the file fetches the
vendor — `drift-collector.test.ts` re-derives live-capability from the emitting
source, so flipping `liveCoverage` without a vendor leg would fail there.
`liveCoverage` stays "none".

Locally, `internal-skills secret-cache resolve "elevenlabs api key"` exited 4
(no match), so the live run was skipped and that fact is recorded in the header.

Also from the R2-R5 voice reviews:
  - the KNOWN DIVERGENCE comment typed the vendor's `detail` as an object; the
    422 validation envelope is an array (`detail: [{ type, loc, msg }]`), the
    401 auth envelope is the object.
  - `bare.start()` moved inside the `try`; `stop()` is guarded by a `started`
    flag because LLMock.stop() throws when unstarted.
  - `mock` is assigned only after `start()` resolves and `afterAll` skips
    `stop()` when it is unset, so a `beforeAll` failure is reported as itself.

Offline: elevenlabs*.drift.ts 12 passed, 1 skipped; drift-collector.test.ts
278 passed.
@jpr5
fix(drift): make the ElevenLabs drift docblocks, titles and DRIFT.md …
69c2bee 
…realtime note truthful

1. elevenlabs.drift.ts music/plan docblock and file header: the case grades
   the mock body against the SDK's MusicPrompt.Raw, but the handler writes the
   test-authored PLAN_FIXTURE verbatim, so the docblock now says plainly that
   this is passthrough conformance against a test-authored fixture with no
   vendor observation, and drops the "before this ..." history.
1b. surface-registry.ts `elevenlabs`: the live /v1/sound-generation case now
   emits a surface-keyed drift block graded against the vendor response
   envelope (status, Content-Type, body presence); the entry gains a
   coverageNote stating that this block alone earns "live" and that music,
   music/stream, the 400 cases and music/plan are offline conformance.
2. elevenlabs.drift.ts 400-case titles: "(vendor: 422 detail[])" replaced with
   "(vendor shape not observed on this route)" — the 422 array envelope was
   observed on Voice Design only.
3. elevenlabs-voice.drift.ts header and the elevenlabs-voice coverageNote:
   1Password / secret-cache maintainer notes removed; state only that the key
   is provided in CI via secrets.ELEVENLABS_API_KEY and that these routes have
   no vendor leg and are offline-only.
4. DRIFT.md Cost: the Realtime probe opens two GA WS connections per run (text
   turn and tool call) and no Beta connection, not "a single GA WS connection".
5. DRIFT.md Gemini Live: the model-name rule is stated as current behaviour
   without the "previously mis-classified ... came to request" history.
6. elevenlabs-voice.drift.ts try/finally comment reworded to what the code
   does: construction above the try, start() and the request inside it.
@pkg-pr-new

pkg-pr-new Bot commented Sep 16, 2026

Copy link
Copy Markdown

Open in StackBlitz

npm i https://pkg.pr.new/@copilotkit/aimock@465

commit: 69c2bee

@jpr5
jpr5 merged commit f9ba33d into main Sep 16, 2026
34 checks passed
@jpr5
jpr5 deleted the docs/s4-drift branch September 16, 2026 21:03
@tylerslaton tylerslaton mentioned this pull request Sep 22, 2026
Merged
tylerslaton added a commit that referenced this pull request Sep 22, 2026
@tylerslaton
chore: release v1.43.0
0b595ed 
## [1.43.0] - 2026-09-22

> **BREAKING** — `aimock -h` is `--help`, not `--host`: `aimock -h 0.0.0.0` exits 1 with `Error:
> Unexpected argument '0.0.0.0'. This command does not take positional arguments`. Migration:
> `--host <string>` (long form only). The `llmock` bin (the Docker ENTRYPOINT) keeps `-h, --host`
> (#453).

### Added

- OpenAI GPT-Live mock, record and offline replay on `GET /v1/live/sessions`, with `onLive` fixtures
  (#468)
- Live sessions enforce message/audio/queue/lifetime limits; recordings sanitize credentials (#468)
- Live drift canaries for lifecycle, audio/transcript, delegation and usage, attributed to
  `openai-live` (#468)
- Chaos `rateLimitRate` / `--chaos-ratelimit`: deterministic 429 with `Retry-After` (#449)
- Chaos `latencyMs` / `--chaos-latency` now actually delays responses on every path (#449)
- OpenAI Files API mock — byte-exact uploads, create-purpose enum, CORS on faults (#445)
- OpenAI fine-tuning jobs mock — deterministic lifecycle, events, cursor pages (#447)
- OpenAI Batches API mock — create/list/retrieve/cancel with real output files (#446)
- `X-Request-Id` echoed or minted on every response; `?requestId=` filters the journal (#450)
- `aimock validate` lints fixture files or directories offline, failing on broken files (#453)
- ElevenLabs Voice Design record/replay — design, save-as-voice, and voice slot management (#452)
- ElevenLabs Voice Design provenance block and strict-mode 503 coverage (#454)

### Changed

- **BREAKING:** `aimock -h` is `--help`, matching `aimock convert -h` and `aimock validate -h`; the
  host override is `--host` only. The `llmock` bin (the Docker entrypoint) keeps its own `-h,
  --host` — see the note above (#453)
- `aimock --config ""`, `--port ""` and `--host ""` are usage errors naming the option (#453)
- Realtime `OpenAI-Beta: realtime=v1` now returns the real sunset rejection, not a session (#461)
- `POST /v1/images/variations` now replays the real removal 404; OpenAI deleted it (#462)
- `ChaosAction` gains `"rateLimit"` — an exhaustive switch over it needs a case (#449)
- `applyChaosAsync()` returns `false | "handled" | "unwritable"` instead of a bare `boolean` (#449)
- Journal `headers` now ALWAYS carry `x-request-id` — exact `toEqual` asserts break (#450)

### Deprecated

- Synchronous `applyChaos()` warns once per process; it skips chaos latency. Use `await
  applyChaosAsync(...)` (#449)

### Fixed

- Reasoning-first chat streams with content and tool calls carry the assistant role in the first
  chunk (#470)
- Journals and metrics reflect delivered vs interrupted responses; one-shot errors never
  double-deliver (#466)
- Nonstreaming OpenAI chat returns block text and tools; text-only blocks finish normally on every
  provider (#467)
- AG-UI drift reads generated 1.0 schemas; `AGUIRunStartedEvent` gains optional `protocolVersion`
  (#469)
- Drift reports list `unverifiedSurfaces`; offline Bedrock/Vertex checks no longer skip silently
  (#460)
- Drift docs match actual coverage; ElevenLabs drift separates vendor observations from fixture
  checks (#465)
- Chaos no longer treats committed headers as a dead response; only status-writing actions are
  skipped (#449)
- Chaos skip logs name what happened; `aimock_chaos_triggered_total` counts only written responses
  (#449)
- A request whose chaos latency was cancelled by client hang-up is no longer served or journalled
  (#449)
- Moderations echoes the request's `model`; default is now `omni-moderation-latest` (#459)
- Image endpoints default to `gpt-image-1` — `dall-e-2`/`dall-e-3` were removed (#459)
- AG-UI record/proxy forwards the caller's headers and raw body upstream; `Accept` is forced to SSE
  (#455)
- The AG-UI recorder refuses to write a fixture from a non-stream or empty 2xx upstream reply (#455)
- A MINTED `x-request-id` is no longer forwarded upstream in record/proxy mode (#450)
MikeRyanDev added a commit that referenced this pull request Sep 22, 2026
@MikeRyanDev
chore: release v1.43.0 (#476)
e863880 
Cuts the accumulated `[Unreleased]` work. **MINOR (1.43.0)**, kept at a
minor bump despite the
`-h` BREAKING banner (#453): the pre-2.0 precedent (1.14.2 shipped a
BREAKING note as a patch)
applies, and the version was agreed ahead of this PR. Do not re-version
to 2.0.0 at review.

**Prepared, not merged.** `publish-release.yml` fires on push-to-main,
so merging this
publishes to npm, tags `v1.43.0`, force-moves `v1`, cuts the GitHub
Release, dispatches the
Docker build, posts to Slack, and runs the PyPI job gated on
`_version.py`. Merge when you
want it live.

## What ships


> **BREAKING** — `aimock -h` is `--help`, not `--host`: `aimock -h
0.0.0.0` exits 1 with `Error: Unexpected argument '0.0.0.0'. This
command does not take positional arguments`. Migration: `--host
<string>` (long form only). The `llmock` bin (the Docker ENTRYPOINT)
keeps `-h, --host` (#453).

### Added

- OpenAI GPT-Live mock, record and offline replay on `GET
/v1/live/sessions`, with `onLive` fixtures (#468)
- Live sessions enforce message/audio/queue/lifetime limits; recordings
sanitize credentials (#468)
- Live drift canaries for lifecycle, audio/transcript, delegation and
usage, attributed to `openai-live` (#468)
- Chaos `rateLimitRate` / `--chaos-ratelimit`: deterministic 429 with
`Retry-After` (#449)
- Chaos `latencyMs` / `--chaos-latency` now actually delays responses on
every path (#449)
- OpenAI Files API mock — byte-exact uploads, create-purpose enum, CORS
on faults (#445)
- OpenAI fine-tuning jobs mock — deterministic lifecycle, events, cursor
pages (#447)
- OpenAI Batches API mock — create/list/retrieve/cancel with real output
files (#446)
- `X-Request-Id` echoed or minted on every response; `?requestId=`
filters the journal (#450)
- `aimock validate` lints fixture files or directories offline, failing
on broken files (#453)
- ElevenLabs Voice Design record/replay — design, save-as-voice, and
voice slot management (#452)
- ElevenLabs Voice Design provenance block and strict-mode 503 coverage
(#454)

### Changed

- **BREAKING:** `aimock -h` is `--help`, matching `aimock convert -h`
and `aimock validate -h`; the host override is `--host` only. The
`llmock` bin (the Docker entrypoint) keeps its own `-h, --host` — see
the note above (#453)
- `aimock --config ""`, `--port ""` and `--host ""` are usage errors
naming the option (#453)
- Realtime `OpenAI-Beta: realtime=v1` now returns the real sunset
rejection, not a session (#461)
- `POST /v1/images/variations` now replays the real removal 404; OpenAI
deleted it (#462)
- `ChaosAction` gains `"rateLimit"` — an exhaustive switch over it needs
a case (#449)
- `applyChaosAsync()` returns `false | "handled" | "unwritable"` instead
of a bare `boolean` (#449)
- Journal `headers` now ALWAYS carry `x-request-id` — exact `toEqual`
asserts break (#450)

### Deprecated

- Synchronous `applyChaos()` warns once per process; it skips chaos
latency. Use `await applyChaosAsync(...)` (#449)

### Fixed

- Reasoning-first chat streams with content and tool calls carry the
assistant role in the first chunk (#470)
- Journals and metrics reflect delivered vs interrupted responses;
one-shot errors never double-deliver (#466)
- Nonstreaming OpenAI chat returns block text and tools; text-only
blocks finish normally on every provider (#467)
- AG-UI drift reads generated 1.0 schemas; `AGUIRunStartedEvent` gains
optional `protocolVersion` (#469)
- Drift reports list `unverifiedSurfaces`; offline Bedrock/Vertex checks
no longer skip silently (#460)
- Drift docs match actual coverage; ElevenLabs drift separates vendor
observations from fixture checks (#465)
- Chaos no longer treats committed headers as a dead response; only
status-writing actions are skipped (#449)
- Chaos skip logs name what happened; `aimock_chaos_triggered_total`
counts only written responses (#449)
- A request whose chaos latency was cancelled by client hang-up is no
longer served or journalled (#449)
- Moderations echoes the request's `model`; default is now
`omni-moderation-latest` (#459)
- Image endpoints default to `gpt-image-1` — `dall-e-2`/`dall-e-3` were
removed (#459)
- AG-UI record/proxy forwards the caller's headers and raw body
upstream; `Accept` is forced to SSE (#455)
- The AG-UI recorder refuses to write a fixture from a non-stream or
empty 2xx upstream reply (#455)
- A MINTED `x-request-id` is no longer forwarded upstream in
record/proxy mode (#450)

The 35 `[Unreleased]` bullets were condensed to 34 one-liners, most ≤100
chars, because
this text is fed verbatim into the GitHub Release body and summarized
into `#oss-alerts`.
Every `(#N)` reference is preserved; the three un-numbered GPT-Live
bullets now cite #468.
Detail lives in the PRs.

## Behaviour changes to read before you merge

- **#453** — `aimock -h` is now `--help`. `aimock -h 0.0.0.0` exits 1.
Migrate to `--host 0.0.0.0`.
The `llmock` bin (Docker `ENTRYPOINT`) is unchanged and still accepts
`-h, --host`.
`--config ""`, `--port ""`, `--host ""` are now usage errors instead of
"not given".
- **#450** — journal `headers` always carry `x-request-id`; exact
`toEqual` asserts on headers break.
- **#449** — `ChaosAction` gains `"rateLimit"` (exhaustive switches need
a case);
`applyChaosAsync()` returns `false | "handled" | "unwritable"`
(truthiness unchanged, explicit
`boolean` bindings need updating); sync `applyChaos()` warns once per
process.
- **#459 / #461 / #462** — defaults track upstream: images default to
`gpt-image-1`, moderations
to `omni-moderation-latest`; `OpenAI-Beta: realtime=v1` returns the real
sunset rejection;
  `POST /v1/images/variations` returns the real removal 404.
- **#466 / #467** — journal and metrics now count interrupted responses
and text-only block
fixtures finish with a normal terminal reason; suites asserting the old
counts or
`finish_reason: "tool_calls"` on text-only blocks will see different
values.

## Version surfaces

Enumerated with `git grep -F 1.42.0`, not from a list. Seven carriers
bumped:

| Surface | Field |
| --- | --- |
| `package.json` | `version` |
| `charts/aimock/Chart.yaml` | `appVersion` |
| `.claude-plugin/plugin.json` | `version` |
| `.claude-plugin/marketplace.json` | `plugins[0].source.version`
(`^1.43.0`) |
| `docs/index.html` | the `aimock v…` banner |
| `packages/aimock-pytest/src/aimock_pytest/_version.py` |
`AIMOCK_VERSION` |
| `packages/aimock-pytest/README.md` | the `--aimock-version` default |

Deliberately untouched: `packages/aimock-pytest/pyproject.toml` (own
PyPI cadence, stays
`0.5.3`), `charts/aimock/Chart.yaml` `version: 0.1.0` (the chart's own
version), the
historical `1.41.0` mentions in `publish-release.yml` and
`npm-publish-verify-workflow.test.ts`, and CHANGELOG history.

`_version.py` is bumped because #466 and #450 change `/__aimock/journal`
and one-shot
`/__aimock/error` delivery — routes the pytest client calls — same rule
as 1.42.0.

## 1.42.1

npm `latest` is currently **1.42.1**, cut from
`maintenance/v1.42.1-lgts-mcp` (#471) and never
merged to main, so main's CHANGELOG had no `[1.42.1]` entry. Its fix
(#470) is on main and ships
here. This PR adds the one-line `## [1.42.1] - 2026-09-18` history entry
so every published
version appears in the file; every prior patch release (1.37.1–1.37.4)
already did.

## README coverage

The Features list had no mention of the OpenAI Files API (#445),
fine-tuning jobs (#447) or
Batches (#446), all shipped in this release — the same gap #443 closed
for the control API.
Added one **OpenAI platform APIs** bullet under Multimedia APIs linking
the Files and fine-tuning
docs pages; Batches has no docs page yet so it is described inline.
`aimock validate`, `--host`,
chaos latency/rate-limit, `X-Request-Id` and GPT-Live were already
covered. The "11 providers
across 22 API surfaces" headline was left as-is: that line enumerates
providers, and re-counting
it is a docs decision rather than a release step.

## npm description sync

Ran the workflow's inline extractor against this branch's README (after
the edit); output is
byte-identical to `package.json.description`, so the on-merge sync is a
no-op.

## Gates

Exit codes captured by redirect to separate files, never through a pipe.
Fresh worktree,
`pnpm install --frozen-lockfile`.

| Gate | Exit |
| --- | --- |
| `pnpm format:check` | 0 |
| `pnpm lint` | 0 |
| `pnpm typecheck` | 0 |
| `pnpm build` | 0 |
| `pnpm test:exports` | 0 |
| `pnpm test` | 0 — 225 files passed; 7341 tests passed |
| `npx commitlint --from origin/main --to HEAD` | 0 (1 cosmetic
`footer-leading-blank` warning) |

🤖 Generated with [Claude Code](https://claude.com/claude-code)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@jpr5