fix(blocks): return text payloads and correct provider completion reasons - #467
Merged
Merged
Conversation
commit: |
Merged
tylerslaton
added a commit
that referenced
this pull request
Sep 22, 2026
## [1.43.0] - 2026-09-22 > **BREAKING** — `aimock -h` is `--help`, not `--host`: `aimock -h 0.0.0.0` exits 1 with `Error: > Unexpected argument '0.0.0.0'. This command does not take positional arguments`. Migration: > `--host <string>` (long form only). The `llmock` bin (the Docker ENTRYPOINT) keeps `-h, --host` > (#453). ### Added - OpenAI GPT-Live mock, record and offline replay on `GET /v1/live/sessions`, with `onLive` fixtures (#468) - Live sessions enforce message/audio/queue/lifetime limits; recordings sanitize credentials (#468) - Live drift canaries for lifecycle, audio/transcript, delegation and usage, attributed to `openai-live` (#468) - Chaos `rateLimitRate` / `--chaos-ratelimit`: deterministic 429 with `Retry-After` (#449) - Chaos `latencyMs` / `--chaos-latency` now actually delays responses on every path (#449) - OpenAI Files API mock — byte-exact uploads, create-purpose enum, CORS on faults (#445) - OpenAI fine-tuning jobs mock — deterministic lifecycle, events, cursor pages (#447) - OpenAI Batches API mock — create/list/retrieve/cancel with real output files (#446) - `X-Request-Id` echoed or minted on every response; `?requestId=` filters the journal (#450) - `aimock validate` lints fixture files or directories offline, failing on broken files (#453) - ElevenLabs Voice Design record/replay — design, save-as-voice, and voice slot management (#452) - ElevenLabs Voice Design provenance block and strict-mode 503 coverage (#454) ### Changed - **BREAKING:** `aimock -h` is `--help`, matching `aimock convert -h` and `aimock validate -h`; the host override is `--host` only. The `llmock` bin (the Docker entrypoint) keeps its own `-h, --host` — see the note above (#453) - `aimock --config ""`, `--port ""` and `--host ""` are usage errors naming the option (#453) - Realtime `OpenAI-Beta: realtime=v1` now returns the real sunset rejection, not a session (#461) - `POST /v1/images/variations` now replays the real removal 404; OpenAI deleted it (#462) - `ChaosAction` gains `"rateLimit"` — an exhaustive switch over it needs a case (#449) - `applyChaosAsync()` returns `false | "handled" | "unwritable"` instead of a bare `boolean` (#449) - Journal `headers` now ALWAYS carry `x-request-id` — exact `toEqual` asserts break (#450) ### Deprecated - Synchronous `applyChaos()` warns once per process; it skips chaos latency. Use `await applyChaosAsync(...)` (#449) ### Fixed - Reasoning-first chat streams with content and tool calls carry the assistant role in the first chunk (#470) - Journals and metrics reflect delivered vs interrupted responses; one-shot errors never double-deliver (#466) - Nonstreaming OpenAI chat returns block text and tools; text-only blocks finish normally on every provider (#467) - AG-UI drift reads generated 1.0 schemas; `AGUIRunStartedEvent` gains optional `protocolVersion` (#469) - Drift reports list `unverifiedSurfaces`; offline Bedrock/Vertex checks no longer skip silently (#460) - Drift docs match actual coverage; ElevenLabs drift separates vendor observations from fixture checks (#465) - Chaos no longer treats committed headers as a dead response; only status-writing actions are skipped (#449) - Chaos skip logs name what happened; `aimock_chaos_triggered_total` counts only written responses (#449) - A request whose chaos latency was cancelled by client hang-up is no longer served or journalled (#449) - Moderations echoes the request's `model`; default is now `omni-moderation-latest` (#459) - Image endpoints default to `gpt-image-1` — `dall-e-2`/`dall-e-3` were removed (#459) - AG-UI record/proxy forwards the caller's headers and raw body upstream; `Accept` is forced to SSE (#455) - The AG-UI recorder refuses to write a fixture from a non-stream or empty 2xx upstream reply (#455) - A MINTED `x-request-id` is no longer forwarded upstream in record/proxy mode (#450)
MikeRyanDev
added a commit
that referenced
this pull request
Sep 22, 2026
Cuts the accumulated `[Unreleased]` work. **MINOR (1.43.0)**, kept at a minor bump despite the `-h` BREAKING banner (#453): the pre-2.0 precedent (1.14.2 shipped a BREAKING note as a patch) applies, and the version was agreed ahead of this PR. Do not re-version to 2.0.0 at review. **Prepared, not merged.** `publish-release.yml` fires on push-to-main, so merging this publishes to npm, tags `v1.43.0`, force-moves `v1`, cuts the GitHub Release, dispatches the Docker build, posts to Slack, and runs the PyPI job gated on `_version.py`. Merge when you want it live. ## What ships > **BREAKING** — `aimock -h` is `--help`, not `--host`: `aimock -h 0.0.0.0` exits 1 with `Error: Unexpected argument '0.0.0.0'. This command does not take positional arguments`. Migration: `--host <string>` (long form only). The `llmock` bin (the Docker ENTRYPOINT) keeps `-h, --host` (#453). ### Added - OpenAI GPT-Live mock, record and offline replay on `GET /v1/live/sessions`, with `onLive` fixtures (#468) - Live sessions enforce message/audio/queue/lifetime limits; recordings sanitize credentials (#468) - Live drift canaries for lifecycle, audio/transcript, delegation and usage, attributed to `openai-live` (#468) - Chaos `rateLimitRate` / `--chaos-ratelimit`: deterministic 429 with `Retry-After` (#449) - Chaos `latencyMs` / `--chaos-latency` now actually delays responses on every path (#449) - OpenAI Files API mock — byte-exact uploads, create-purpose enum, CORS on faults (#445) - OpenAI fine-tuning jobs mock — deterministic lifecycle, events, cursor pages (#447) - OpenAI Batches API mock — create/list/retrieve/cancel with real output files (#446) - `X-Request-Id` echoed or minted on every response; `?requestId=` filters the journal (#450) - `aimock validate` lints fixture files or directories offline, failing on broken files (#453) - ElevenLabs Voice Design record/replay — design, save-as-voice, and voice slot management (#452) - ElevenLabs Voice Design provenance block and strict-mode 503 coverage (#454) ### Changed - **BREAKING:** `aimock -h` is `--help`, matching `aimock convert -h` and `aimock validate -h`; the host override is `--host` only. The `llmock` bin (the Docker entrypoint) keeps its own `-h, --host` — see the note above (#453) - `aimock --config ""`, `--port ""` and `--host ""` are usage errors naming the option (#453) - Realtime `OpenAI-Beta: realtime=v1` now returns the real sunset rejection, not a session (#461) - `POST /v1/images/variations` now replays the real removal 404; OpenAI deleted it (#462) - `ChaosAction` gains `"rateLimit"` — an exhaustive switch over it needs a case (#449) - `applyChaosAsync()` returns `false | "handled" | "unwritable"` instead of a bare `boolean` (#449) - Journal `headers` now ALWAYS carry `x-request-id` — exact `toEqual` asserts break (#450) ### Deprecated - Synchronous `applyChaos()` warns once per process; it skips chaos latency. Use `await applyChaosAsync(...)` (#449) ### Fixed - Reasoning-first chat streams with content and tool calls carry the assistant role in the first chunk (#470) - Journals and metrics reflect delivered vs interrupted responses; one-shot errors never double-deliver (#466) - Nonstreaming OpenAI chat returns block text and tools; text-only blocks finish normally on every provider (#467) - AG-UI drift reads generated 1.0 schemas; `AGUIRunStartedEvent` gains optional `protocolVersion` (#469) - Drift reports list `unverifiedSurfaces`; offline Bedrock/Vertex checks no longer skip silently (#460) - Drift docs match actual coverage; ElevenLabs drift separates vendor observations from fixture checks (#465) - Chaos no longer treats committed headers as a dead response; only status-writing actions are skipped (#449) - Chaos skip logs name what happened; `aimock_chaos_triggered_total` counts only written responses (#449) - A request whose chaos latency was cancelled by client hang-up is no longer served or journalled (#449) - Moderations echoes the request's `model`; default is now `omni-moderation-latest` (#459) - Image endpoints default to `gpt-image-1` — `dall-e-2`/`dall-e-3` were removed (#459) - AG-UI record/proxy forwards the caller's headers and raw body upstream; `Accept` is forced to SSE (#455) - The AG-UI recorder refuses to write a fixture from a non-stream or empty 2xx upstream reply (#455) - A MINTED `x-request-id` is no longer forwarded upstream in record/proxy mode (#450) The 35 `[Unreleased]` bullets were condensed to 34 one-liners, most ≤100 chars, because this text is fed verbatim into the GitHub Release body and summarized into `#oss-alerts`. Every `(#N)` reference is preserved; the three un-numbered GPT-Live bullets now cite #468. Detail lives in the PRs. ## Behaviour changes to read before you merge - **#453** — `aimock -h` is now `--help`. `aimock -h 0.0.0.0` exits 1. Migrate to `--host 0.0.0.0`. The `llmock` bin (Docker `ENTRYPOINT`) is unchanged and still accepts `-h, --host`. `--config ""`, `--port ""`, `--host ""` are now usage errors instead of "not given". - **#450** — journal `headers` always carry `x-request-id`; exact `toEqual` asserts on headers break. - **#449** — `ChaosAction` gains `"rateLimit"` (exhaustive switches need a case); `applyChaosAsync()` returns `false | "handled" | "unwritable"` (truthiness unchanged, explicit `boolean` bindings need updating); sync `applyChaos()` warns once per process. - **#459 / #461 / #462** — defaults track upstream: images default to `gpt-image-1`, moderations to `omni-moderation-latest`; `OpenAI-Beta: realtime=v1` returns the real sunset rejection; `POST /v1/images/variations` returns the real removal 404. - **#466 / #467** — journal and metrics now count interrupted responses and text-only block fixtures finish with a normal terminal reason; suites asserting the old counts or `finish_reason: "tool_calls"` on text-only blocks will see different values. ## Version surfaces Enumerated with `git grep -F 1.42.0`, not from a list. Seven carriers bumped: | Surface | Field | | --- | --- | | `package.json` | `version` | | `charts/aimock/Chart.yaml` | `appVersion` | | `.claude-plugin/plugin.json` | `version` | | `.claude-plugin/marketplace.json` | `plugins[0].source.version` (`^1.43.0`) | | `docs/index.html` | the `aimock v…` banner | | `packages/aimock-pytest/src/aimock_pytest/_version.py` | `AIMOCK_VERSION` | | `packages/aimock-pytest/README.md` | the `--aimock-version` default | Deliberately untouched: `packages/aimock-pytest/pyproject.toml` (own PyPI cadence, stays `0.5.3`), `charts/aimock/Chart.yaml` `version: 0.1.0` (the chart's own version), the historical `1.41.0` mentions in `publish-release.yml` and `npm-publish-verify-workflow.test.ts`, and CHANGELOG history. `_version.py` is bumped because #466 and #450 change `/__aimock/journal` and one-shot `/__aimock/error` delivery — routes the pytest client calls — same rule as 1.42.0. ## 1.42.1 npm `latest` is currently **1.42.1**, cut from `maintenance/v1.42.1-lgts-mcp` (#471) and never merged to main, so main's CHANGELOG had no `[1.42.1]` entry. Its fix (#470) is on main and ships here. This PR adds the one-line `## [1.42.1] - 2026-09-18` history entry so every published version appears in the file; every prior patch release (1.37.1–1.37.4) already did. ## README coverage The Features list had no mention of the OpenAI Files API (#445), fine-tuning jobs (#447) or Batches (#446), all shipped in this release — the same gap #443 closed for the control API. Added one **OpenAI platform APIs** bullet under Multimedia APIs linking the Files and fine-tuning docs pages; Batches has no docs page yet so it is described inline. `aimock validate`, `--host`, chaos latency/rate-limit, `X-Request-Id` and GPT-Live were already covered. The "11 providers across 22 API surfaces" headline was left as-is: that line enumerates providers, and re-counting it is a docs decision rather than a release step. ## npm description sync Ran the workflow's inline extractor against this branch's README (after the edit); output is byte-identical to `package.json.description`, so the on-merge sync is a no-op. ## Gates Exit codes captured by redirect to separate files, never through a pipe. Fresh worktree, `pnpm install --frozen-lockfile`. | Gate | Exit | | --- | --- | | `pnpm format:check` | 0 | | `pnpm lint` | 0 | | `pnpm typecheck` | 0 | | `pnpm build` | 0 | | `pnpm test:exports` | 0 | | `pnpm test` | 0 — 225 files passed; 7341 tests passed | | `npx commitlint --from origin/main --to HEAD` | 0 (1 cosmetic `footer-leading-blank` warning) | 🤖 Generated with [Claude Code](https://claude.com/claude-code)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Text-only
blocksfixtures currently end with a tool-call terminal even though no tools were emitted. Nonstreaming OpenAI chat also drops block content and tools entirely. This change derives block-path defaults from the resolved output: text finishes normally, tool-containing blocks retain tool-call terminals, and nonstreaming OpenAI chat returns the authored block payload.The shared outcome feeds both streaming and nonstreaming paths for OpenAI chat, Claude messages, Gemini, Gemini Interactions, Cohere, Bedrock invoke, and Bedrock Converse. Existing supported finish overrides, legacy fallback for absent/empty blocks, block order, and provider-supported tool IDs remain intact. Nonempty blocks remain authoritative over conflicting legacy fields. Interactions retains its existing ignored finish-override behavior.
Local real red–green proof
Baseline:
b1ba430658c0fac66dcf24464e7d44178553e477.Verified candidate:
5c3d27461a038b7e6aa055f295ab3782f1991966(treef70d3873d3d35717437ef35c3b12e4f28eacfa23).The same frozen probe exercised 96 actual HTTP responses from locally running LLMock: eight protocols including unchanged Ollama controls, both response modes, and text/tool/mixed/legacy/override/conflicting-legacy fixtures. JSON, SSE, NDJSON, and AWS EventStream responses were decoded from the wire; no fake responses or external LLM calls were used.
Commands run before production changes, from the integration checkout at the baseline:
Commands run independently after integration, from that checkout at the candidate:
Build and probe commands exited 0 in both runs. The exact 33 failing cells flipped; all 63 passing controls stayed passing. Probe and checker were byte-identical between runs, verified against these SHA-256 hashes:
Actual text-only wire comparisons:
content="",finish_reason="tool_calls"content="BLOCK_TEXT",finish_reason="stop"stop_reason="tool_use"stop_reason="end_turn"finishReason="FUNCTION_CALL"finishReason="STOP"finish_reason="TOOL_CALL"finish_reason="COMPLETE"stop_reason="tool_use"stop_reason="end_turn"The standalone probe paths above are local evidence artifacts, not repository files. Full responses and cell comparison are retained under
~/.local/share/copilotkit/cr/aimock-section1/{repro,integrated-value}/. Repository-contained HTTP regressions are reproducible from the candidate checkout:Each new suite was observed failing against its pre-fix implementation before passing after the fix. Controls cover empty text, both mixed orders, trailing empty text, tool arguments/IDs, overrides, legacy fallback, and payload plus terminal together.
Review and validation
All 17 valid review reports were adjudicated with zero mandatory findings. The independent promotion audit recommended zero promotions; no source changes or further confirmation round were required. Separate maintenance items are recorded in the review follow-ups.
Final pre-push checks passed in order, all exit 0:
pnpm run format:check pnpm run lint pnpm run typecheck pnpm run test --maxWorkers=1 --no-file-parallelism pnpm run buildThe full suite passed 6,974 tests in 213 files, with zero skips or unhandled errors. The initial default-concurrency run had four timing failures and two worker RPC timeouts while host load exceeded 46. All four failed cases passed once in isolation with unchanged assertions and timeouts; the full gate then restarted with one worker to reduce contention. No source fixes, timeout changes, or test exclusions were used.
The four commits already separate the shared OpenAI change and provider adaptations by concern; all commit messages pass commitlint. The checked HEAD and tree above remain unchanged and clean. All 30 CI checks passed on
5c3d27461a038b7e6aa055f295ab3782f1991966, including the supported Node test matrix, Python tests, static quality, package exports, and Docker build.