feat: mock, record and replay GPT-Live WebSocket sessions - #468
Merged
Merged
Conversation
commit: |
Merged
tylerslaton
added a commit
that referenced
this pull request
Sep 22, 2026
## [1.43.0] - 2026-09-22 > **BREAKING** — `aimock -h` is `--help`, not `--host`: `aimock -h 0.0.0.0` exits 1 with `Error: > Unexpected argument '0.0.0.0'. This command does not take positional arguments`. Migration: > `--host <string>` (long form only). The `llmock` bin (the Docker ENTRYPOINT) keeps `-h, --host` > (#453). ### Added - OpenAI GPT-Live mock, record and offline replay on `GET /v1/live/sessions`, with `onLive` fixtures (#468) - Live sessions enforce message/audio/queue/lifetime limits; recordings sanitize credentials (#468) - Live drift canaries for lifecycle, audio/transcript, delegation and usage, attributed to `openai-live` (#468) - Chaos `rateLimitRate` / `--chaos-ratelimit`: deterministic 429 with `Retry-After` (#449) - Chaos `latencyMs` / `--chaos-latency` now actually delays responses on every path (#449) - OpenAI Files API mock — byte-exact uploads, create-purpose enum, CORS on faults (#445) - OpenAI fine-tuning jobs mock — deterministic lifecycle, events, cursor pages (#447) - OpenAI Batches API mock — create/list/retrieve/cancel with real output files (#446) - `X-Request-Id` echoed or minted on every response; `?requestId=` filters the journal (#450) - `aimock validate` lints fixture files or directories offline, failing on broken files (#453) - ElevenLabs Voice Design record/replay — design, save-as-voice, and voice slot management (#452) - ElevenLabs Voice Design provenance block and strict-mode 503 coverage (#454) ### Changed - **BREAKING:** `aimock -h` is `--help`, matching `aimock convert -h` and `aimock validate -h`; the host override is `--host` only. The `llmock` bin (the Docker entrypoint) keeps its own `-h, --host` — see the note above (#453) - `aimock --config ""`, `--port ""` and `--host ""` are usage errors naming the option (#453) - Realtime `OpenAI-Beta: realtime=v1` now returns the real sunset rejection, not a session (#461) - `POST /v1/images/variations` now replays the real removal 404; OpenAI deleted it (#462) - `ChaosAction` gains `"rateLimit"` — an exhaustive switch over it needs a case (#449) - `applyChaosAsync()` returns `false | "handled" | "unwritable"` instead of a bare `boolean` (#449) - Journal `headers` now ALWAYS carry `x-request-id` — exact `toEqual` asserts break (#450) ### Deprecated - Synchronous `applyChaos()` warns once per process; it skips chaos latency. Use `await applyChaosAsync(...)` (#449) ### Fixed - Reasoning-first chat streams with content and tool calls carry the assistant role in the first chunk (#470) - Journals and metrics reflect delivered vs interrupted responses; one-shot errors never double-deliver (#466) - Nonstreaming OpenAI chat returns block text and tools; text-only blocks finish normally on every provider (#467) - AG-UI drift reads generated 1.0 schemas; `AGUIRunStartedEvent` gains optional `protocolVersion` (#469) - Drift reports list `unverifiedSurfaces`; offline Bedrock/Vertex checks no longer skip silently (#460) - Drift docs match actual coverage; ElevenLabs drift separates vendor observations from fixture checks (#465) - Chaos no longer treats committed headers as a dead response; only status-writing actions are skipped (#449) - Chaos skip logs name what happened; `aimock_chaos_triggered_total` counts only written responses (#449) - A request whose chaos latency was cancelled by client hang-up is no longer served or journalled (#449) - Moderations echoes the request's `model`; default is now `omni-moderation-latest` (#459) - Image endpoints default to `gpt-image-1` — `dall-e-2`/`dall-e-3` were removed (#459) - AG-UI record/proxy forwards the caller's headers and raw body upstream; `Accept` is forced to SSE (#455) - The AG-UI recorder refuses to write a fixture from a non-stream or empty 2xx upstream reply (#455) - A MINTED `x-request-id` is no longer forwarded upstream in record/proxy mode (#450)
MikeRyanDev
added a commit
that referenced
this pull request
Sep 22, 2026
Cuts the accumulated `[Unreleased]` work. **MINOR (1.43.0)**, kept at a minor bump despite the `-h` BREAKING banner (#453): the pre-2.0 precedent (1.14.2 shipped a BREAKING note as a patch) applies, and the version was agreed ahead of this PR. Do not re-version to 2.0.0 at review. **Prepared, not merged.** `publish-release.yml` fires on push-to-main, so merging this publishes to npm, tags `v1.43.0`, force-moves `v1`, cuts the GitHub Release, dispatches the Docker build, posts to Slack, and runs the PyPI job gated on `_version.py`. Merge when you want it live. ## What ships > **BREAKING** — `aimock -h` is `--help`, not `--host`: `aimock -h 0.0.0.0` exits 1 with `Error: Unexpected argument '0.0.0.0'. This command does not take positional arguments`. Migration: `--host <string>` (long form only). The `llmock` bin (the Docker ENTRYPOINT) keeps `-h, --host` (#453). ### Added - OpenAI GPT-Live mock, record and offline replay on `GET /v1/live/sessions`, with `onLive` fixtures (#468) - Live sessions enforce message/audio/queue/lifetime limits; recordings sanitize credentials (#468) - Live drift canaries for lifecycle, audio/transcript, delegation and usage, attributed to `openai-live` (#468) - Chaos `rateLimitRate` / `--chaos-ratelimit`: deterministic 429 with `Retry-After` (#449) - Chaos `latencyMs` / `--chaos-latency` now actually delays responses on every path (#449) - OpenAI Files API mock — byte-exact uploads, create-purpose enum, CORS on faults (#445) - OpenAI fine-tuning jobs mock — deterministic lifecycle, events, cursor pages (#447) - OpenAI Batches API mock — create/list/retrieve/cancel with real output files (#446) - `X-Request-Id` echoed or minted on every response; `?requestId=` filters the journal (#450) - `aimock validate` lints fixture files or directories offline, failing on broken files (#453) - ElevenLabs Voice Design record/replay — design, save-as-voice, and voice slot management (#452) - ElevenLabs Voice Design provenance block and strict-mode 503 coverage (#454) ### Changed - **BREAKING:** `aimock -h` is `--help`, matching `aimock convert -h` and `aimock validate -h`; the host override is `--host` only. The `llmock` bin (the Docker entrypoint) keeps its own `-h, --host` — see the note above (#453) - `aimock --config ""`, `--port ""` and `--host ""` are usage errors naming the option (#453) - Realtime `OpenAI-Beta: realtime=v1` now returns the real sunset rejection, not a session (#461) - `POST /v1/images/variations` now replays the real removal 404; OpenAI deleted it (#462) - `ChaosAction` gains `"rateLimit"` — an exhaustive switch over it needs a case (#449) - `applyChaosAsync()` returns `false | "handled" | "unwritable"` instead of a bare `boolean` (#449) - Journal `headers` now ALWAYS carry `x-request-id` — exact `toEqual` asserts break (#450) ### Deprecated - Synchronous `applyChaos()` warns once per process; it skips chaos latency. Use `await applyChaosAsync(...)` (#449) ### Fixed - Reasoning-first chat streams with content and tool calls carry the assistant role in the first chunk (#470) - Journals and metrics reflect delivered vs interrupted responses; one-shot errors never double-deliver (#466) - Nonstreaming OpenAI chat returns block text and tools; text-only blocks finish normally on every provider (#467) - AG-UI drift reads generated 1.0 schemas; `AGUIRunStartedEvent` gains optional `protocolVersion` (#469) - Drift reports list `unverifiedSurfaces`; offline Bedrock/Vertex checks no longer skip silently (#460) - Drift docs match actual coverage; ElevenLabs drift separates vendor observations from fixture checks (#465) - Chaos no longer treats committed headers as a dead response; only status-writing actions are skipped (#449) - Chaos skip logs name what happened; `aimock_chaos_triggered_total` counts only written responses (#449) - A request whose chaos latency was cancelled by client hang-up is no longer served or journalled (#449) - Moderations echoes the request's `model`; default is now `omni-moderation-latest` (#459) - Image endpoints default to `gpt-image-1` — `dall-e-2`/`dall-e-3` were removed (#459) - AG-UI record/proxy forwards the caller's headers and raw body upstream; `Accept` is forced to SSE (#455) - The AG-UI recorder refuses to write a fixture from a non-stream or empty 2xx upstream reply (#455) - A MINTED `x-request-id` is no longer forwarded upstream in record/proxy mode (#450) The 35 `[Unreleased]` bullets were condensed to 34 one-liners, most ≤100 chars, because this text is fed verbatim into the GitHub Release body and summarized into `#oss-alerts`. Every `(#N)` reference is preserved; the three un-numbered GPT-Live bullets now cite #468. Detail lives in the PRs. ## Behaviour changes to read before you merge - **#453** — `aimock -h` is now `--help`. `aimock -h 0.0.0.0` exits 1. Migrate to `--host 0.0.0.0`. The `llmock` bin (Docker `ENTRYPOINT`) is unchanged and still accepts `-h, --host`. `--config ""`, `--port ""`, `--host ""` are now usage errors instead of "not given". - **#450** — journal `headers` always carry `x-request-id`; exact `toEqual` asserts on headers break. - **#449** — `ChaosAction` gains `"rateLimit"` (exhaustive switches need a case); `applyChaosAsync()` returns `false | "handled" | "unwritable"` (truthiness unchanged, explicit `boolean` bindings need updating); sync `applyChaos()` warns once per process. - **#459 / #461 / #462** — defaults track upstream: images default to `gpt-image-1`, moderations to `omni-moderation-latest`; `OpenAI-Beta: realtime=v1` returns the real sunset rejection; `POST /v1/images/variations` returns the real removal 404. - **#466 / #467** — journal and metrics now count interrupted responses and text-only block fixtures finish with a normal terminal reason; suites asserting the old counts or `finish_reason: "tool_calls"` on text-only blocks will see different values. ## Version surfaces Enumerated with `git grep -F 1.42.0`, not from a list. Seven carriers bumped: | Surface | Field | | --- | --- | | `package.json` | `version` | | `charts/aimock/Chart.yaml` | `appVersion` | | `.claude-plugin/plugin.json` | `version` | | `.claude-plugin/marketplace.json` | `plugins[0].source.version` (`^1.43.0`) | | `docs/index.html` | the `aimock v…` banner | | `packages/aimock-pytest/src/aimock_pytest/_version.py` | `AIMOCK_VERSION` | | `packages/aimock-pytest/README.md` | the `--aimock-version` default | Deliberately untouched: `packages/aimock-pytest/pyproject.toml` (own PyPI cadence, stays `0.5.3`), `charts/aimock/Chart.yaml` `version: 0.1.0` (the chart's own version), the historical `1.41.0` mentions in `publish-release.yml` and `npm-publish-verify-workflow.test.ts`, and CHANGELOG history. `_version.py` is bumped because #466 and #450 change `/__aimock/journal` and one-shot `/__aimock/error` delivery — routes the pytest client calls — same rule as 1.42.0. ## 1.42.1 npm `latest` is currently **1.42.1**, cut from `maintenance/v1.42.1-lgts-mcp` (#471) and never merged to main, so main's CHANGELOG had no `[1.42.1]` entry. Its fix (#470) is on main and ships here. This PR adds the one-line `## [1.42.1] - 2026-09-18` history entry so every published version appears in the file; every prior patch release (1.37.1–1.37.4) already did. ## README coverage The Features list had no mention of the OpenAI Files API (#445), fine-tuning jobs (#447) or Batches (#446), all shipped in this release — the same gap #443 closed for the control API. Added one **OpenAI platform APIs** bullet under Multimedia APIs linking the Files and fine-tuning docs pages; Batches has no docs page yet so it is described inline. `aimock validate`, `--host`, chaos latency/rate-limit, `X-Request-Id` and GPT-Live were already covered. The "11 providers across 22 API surfaces" headline was left as-is: that line enumerates providers, and re-counting it is a docs decision rather than a release step. ## npm description sync Ran the workflow's inline extractor against this branch's README (after the edit); output is byte-identical to `package.json.description`, so the on-merge sync is a no-op. ## Gates Exit codes captured by redirect to separate files, never through a pipe. Fresh worktree, `pnpm install --frozen-lockfile`. | Gate | Exit | | --- | --- | | `pnpm format:check` | 0 | | `pnpm lint` | 0 | | `pnpm typecheck` | 0 | | `pnpm build` | 0 | | `pnpm test:exports` | 0 | | `pnpm test` | 0 — 225 files passed; 7341 tests passed | | `npx commitlint --from origin/main --to HEAD` | 0 (1 cosmetic `footer-leading-blank` warning) | 🤖 Generated with [Claude Code](https://claude.com/claude-code)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add GPT-Live support for the primary
/v1/live/sessionsWebSocket in client and managed backend modes. Live fixtures previously failed registration; applications can now useonLiveor file fixtures, record provider sessions, and replay them offline with audio, transcripts, delegation and managed tool continuation.Replay preserves causal ordering, timing and equivalent PCM rechunking, with fresh identifiers per connection. Includes bounded resources, credential sanitation, cleanup, Jest/Vitest helpers, examples and provider drift canaries. Surgical fixes preserve helper limits, accept empty PCM and admitted timeout values, and prevent credential export while retaining harmless malformed URLs.
Approved plan. Primary WebSocket only; WebRTC/SDP and sideband are excluded. Audio comes from fixtures.
Local red–green proof
Identical repro inputs ran before and after each fix against real public LLMock sockets/recording exports. Counts below cover selected cases; filtered tests are not counted as passes.
response is not a recognized typefixture-mismatchfixtures:0,unsafe-exportfixtures:1, byte-for-byte preservation and offline replayJournal.onAddexposed owner secretsfixtures:1,recoverableCredentialPersisted:truefixtures:0,recoverableCredentialPersisted:falseProvider acceptance and verification
Provider usage snapshots are not a complete billing total; an earlier incomplete recording has unmeasured usage.
CI: 31 checks passed, 1 failed, 2 skipped. The live-drift check fails while collecting the existing main baseline: upstream AG-UI moved
core/src/types.tstocore/src/generated/types.ts. The PR-head provider leg and delta gate are skipped. This requires a separate baseline collector compatibility fix; an unchanged rerun will remain blocked.