Skip to content

examples: validate stream resume prerequisites - #516

Merged
jbeckwith-oai merged 1 commit into
mainfrom
codex/streaming-previous-response-prerequisites
Aug 24, 2026
Merged

examples: validate stream resume prerequisites#516
jbeckwith-oai merged 1 commit into
mainfrom
codex/streaming-previous-response-prerequisites

Conversation

@jbeckwith-oai

Copy link
Copy Markdown
Contributor

Summary

  • Fail fast when either previous-response streaming example receives no initial events, before dereferencing the final event or sleeping.
  • Require a captured, nonempty response ID in both the plain and structured examples before issuing a resumed request.
  • Preserve successful resume sequencing and structured parsed-output validation with deterministic, network-free regression coverage.

Evidence and scope

  • Before the fix, the new regression suite produced six errors: empty initial streams raised NoMethodError, while missing created events or empty response IDs attempted an invalid resumed request.
  • The existing successful plain/structured resume characterization passed before and after the fix.
  • Blast radius: one handwritten, generator-exempt example and its own new example-specific test file; no generated SDK code, shared streaming/transport behavior, API contracts, dependencies, or serialization changes.
  • Generator ownership: CONTRIBUTING.md explicitly identifies examples/ as handwritten and never modified by the generator; wholly handwritten files are outside the Castiron custom-code budget.
  • Compatibility: no public SDK/API changes; the successful example flows and parsed-output validation are unchanged.

Verification

  • Ruby 4.0.6 focused regression suite: bundle exec ruby test/openai/streaming_previous_response_example_test.rb — 7 tests, 52 assertions.
  • Ruby 3.3.12 and 3.4.10 focused regression suites: the same 7 tests and 52 assertions pass on both supported minors.
  • Responses streaming subsystem: bundle exec ruby test/openai/resources/responses/streaming_test.rb — 25 tests, 95 assertions.
  • Existing handwritten-example characterization: bundle exec ruby test/openai/structured_outputs_responses_function_calling_example_test.rb — 3 tests, 17 assertions.
  • Full suite: TMPDIR=/private/tmp ./scripts/test — 1,198 tests, 10,645 assertions, 0 failures/errors, 1 existing skip. Canonical TMPDIR avoids an unrelated existing macOS /var versus /private/var expectation in FastFormatTest without changing repository code.
  • Complete lint/type checks: bundle exec rake lint — Sorbet clean, 1,239 RBS files validated, and 2,747 Ruby files inspected with no RuboCop offenses.
  • Offline example inventory: bundle exec rake test:examples:inventory.
  • Castiron custom-code checker regression suite: python3 -m unittest discover -s scripts/castiron -p 'test_custom_code*.py' — 51 tests passed, 1 existing skip.
  • Trusted committed-head Castiron budget check: python3 -B -I scripts/castiron/custom_code_budget.py check --repo /path/to/sdk --base edfb30b2973d68b4a19ce3f86f1de7313f730702 --head 579672e64a25c4c94fb068abe3ed4a65974a40db --public --fetch --out /tmp/custom-code-budget — isolation passed; 2,659 custom lines under the 4,000-line limit, with 1,341 lines of headroom.
  • Focused Ruby formatting and RuboCop checks, git diff --check, exact-base path/scope review, extensive general review, and thermo-nuclear maintainability review.

@jbeckwith-oai jbeckwith-oai added the codex-maintenance Low-risk maintenance changes created by Codex label Aug 23, 2026
@github-actions

github-actions Bot commented Aug 23, 2026

Copy link
Copy Markdown
Contributor

Castiron custom code

✅ No new custom-code files detected.

48 mixed files remain; 0 existing customizations changed.

Compared edfb30b2973d579672e64a25. Generated baselines verified.

48 existing customizations unchanged
  • lib/openai.rb
  • lib/openai/client.rb
  • lib/openai/models/chat/chat_completion_message.rb
  • lib/openai/models/chat/chat_completion_message_function_tool_call.rb
  • lib/openai/models/chat/completion_create_params.rb
  • lib/openai/models/response_format_json_schema.rb
  • lib/openai/models/responses/function_tool.rb
  • lib/openai/models/responses/response.rb
  • lib/openai/models/responses/response_create_params.rb
  • lib/openai/models/responses/response_format_text_config.rb
  • lib/openai/models/responses/response_format_text_json_schema_config.rb
  • lib/openai/models/responses/response_function_tool_call.rb
  • lib/openai/models/responses/response_function_web_search.rb
  • lib/openai/models/responses/response_output_text.rb
  • lib/openai/models/responses/tool.rb
  • lib/openai/resources/beta/responses.rb
  • lib/openai/resources/beta/threads.rb
  • lib/openai/resources/chat/completions.rb
  • lib/openai/resources/files.rb
  • lib/openai/resources/responses.rb
  • lib/openai/resources/vector_stores/file_batches.rb
  • lib/openai/resources/vector_stores/files.rb
  • lib/openai/resources/webhooks.rb
  • rbi/openai/client.rbi
  • rbi/openai/models/chat/chat_completion_message.rbi
  • rbi/openai/models/chat/chat_completion_message_function_tool_call.rbi
  • rbi/openai/models/chat/completion_create_params.rbi
  • rbi/openai/models/response_format_json_schema.rbi
  • rbi/openai/models/responses/response.rbi
  • rbi/openai/models/responses/response_create_params.rbi
  • rbi/openai/models/responses/response_function_tool_call.rbi
  • rbi/openai/models/responses/response_function_web_search.rbi
  • rbi/openai/models/responses/response_output_text.rbi
  • rbi/openai/resources/chat/completions.rbi
  • rbi/openai/resources/files.rbi
  • rbi/openai/resources/responses.rbi
  • rbi/openai/resources/vector_stores/file_batches.rbi
  • rbi/openai/resources/vector_stores/files.rbi
  • scripts/castiron/README.md
  • scripts/castiron/custom_code_report.py

8 more in the full report.

A changed generated baseline means this report cannot reliably identify which handwritten lines changed.

Inspect the custom-code diff

Download the exact patch produced by this run (requires repository access):

gh run download 32748376996 --repo openai/openai-ruby \
  --name castiron-custom-code-32748376996-1 --dir /tmp/castiron-custom-code-32748376996-1
git apply --stat /tmp/castiron-custom-code-32748376996-1/custom-code.patch
cat /tmp/castiron-custom-code-32748376996-1/custom-code.patch

Or reproduce it from an SDK checkout containing the vendored reporter:

git fetch --no-tags origin edfb30b2973d68b4a19ce3f86f1de7313f730702 579672e64a25c4c94fb068abe3ed4a65974a40db
python3 scripts/castiron/custom_code_report.py report \
  --base edfb30b2973d68b4a19ce3f86f1de7313f730702 \
  --head 579672e64a25c4c94fb068abe3ed4a65974a40db --fetch --require-head-hash --public \
  --out /tmp/castiron-custom-code-579672e64a25
cat /tmp/castiron-custom-code-579672e64a25/custom-code.patch

This is the current full custom patch for mixed files, not an attribution of only the handwritten lines changed by this PR.

Full report and patch

@jbeckwith-oai
jbeckwith-oai marked this pull request as ready for review August 23, 2026 19:47
@jbeckwith-oai
jbeckwith-oai requested a review from a team as a code owner August 23, 2026 19:47
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Aug 23, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-08-23T19:49:08.807066Z 579672e Draft marked ready
🔒 Security Review Completed 2026-08-23T19:49:27.855342Z 579672e Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@openai-sdks

openai-sdks Bot commented Aug 23, 2026

Copy link
Copy Markdown
Contributor

OkTest Summary

237/237 SDK tests passed in 7.919s for Ruby SDK PR #516.

Test results — 42 files
Test Result Time
tests/chat-completions-complex-body.test.ts ✅ Passed 116ms
tests/chat-completions-create.test.ts ✅ Passed 179ms
tests/chat-completions-stream.test.ts ✅ Passed 106ms
tests/files-content-binary.test.ts ✅ Passed 104ms
tests/files-create-multipart.test.ts ✅ Passed 133ms
tests/files-list-pagination.test.ts ✅ Passed 137ms
tests/initialize-config.test.ts ✅ Passed 155ms
tests/instance-isolation.test.ts ✅ Passed 138ms
tests/models-list.test.ts ✅ Passed 185ms
tests/responses-background-lifecycle.test.ts ✅ Passed 173ms
tests/responses-body-method-errors.test.ts ✅ Passed 334ms
tests/responses-cancel-timeout.test.ts ✅ Passed 218ms
tests/responses-cancel.test.ts ✅ Passed 165ms
tests/responses-compact-retries.test.ts ✅ Passed 176ms
tests/responses-compact.test.ts ✅ Passed 180ms
tests/responses-create-advanced-stream.test.ts ✅ Passed 119ms
tests/responses-create-advanced.test.ts ✅ Passed 165ms
tests/responses-create-disconnect.test.ts ✅ Passed 965ms
tests/responses-create-errors.test.ts ✅ Passed 223ms
tests/responses-create-malformed-api-responses.test.ts ✅ Passed 97ms
tests/responses-create-retries.test.ts ✅ Passed 198ms
tests/responses-create-stream-failures.test.ts ✅ Passed 102ms
tests/responses-create-stream-timeout.test.ts ✅ Passed 190ms
tests/responses-create-stream-wire.test.ts ✅ Passed 1.968s
tests/responses-create-stream.test.ts ✅ Passed 113ms
tests/responses-create-terminal-states.test.ts ✅ Passed 213ms
tests/responses-create-timeout.test.ts ✅ Passed 203ms
tests/responses-create.test.ts ✅ Passed 216ms
tests/responses-delete.test.ts ✅ Passed 187ms
tests/responses-input-items-errors.test.ts ✅ Passed 142ms
tests/responses-input-items-list.test.ts ✅ Passed 141ms
tests/responses-input-items-options.test.ts ✅ Passed 111ms
tests/responses-input-tokens-count-timeout.test.ts ✅ Passed 195ms
tests/responses-input-tokens-count.test.ts ✅ Passed 156ms
tests/responses-malformed-inputs.test.ts ✅ Passed 1.557s
tests/responses-not-found-errors.test.ts ✅ Passed 228ms
tests/responses-parse.test.ts ✅ Passed 148ms
tests/responses-retrieve-retries.test.ts ✅ Passed 258ms
tests/responses-retrieve.test.ts ✅ Passed 188ms
tests/responses-stored-method-errors.test.ts ✅ Passed 485ms
tests/retry-behavior.test.ts ✅ Passed 3s
tests/sdk-error-shape.test.ts ✅ Passed 306ms

View OkTest run #32662384022

SDK merge (3210f6a22ec5) · head (579672e64a25) · base (edfb30b2973d) · OkTest (2b1bdfd25e98)

@apcha-oai apcha-oai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[sdk codex review] Approved at 579672e64a25c4c94fb068abe3ed4a65974a40db after two independent, full code reviews and reconciliation of existing feedback. No actionable findings.

The plain and structured flows now reject empty initial streams and missing/empty captured response IDs before dereferencing, waiting, or resuming. All seven regression/characterization cases were examined, including successful resume IDs, cursors, and structured output. The successful behavior and public SDK surface are unchanged.

Existing hosted CI for the reviewed head merged with edfb30b2973d68b4a19ce3f86f1de7313f730702 passed on Ruby 3.3, 3.4, and 4.0: 1,198 tests and 10,646 assertions per version, with no failures, errors, or skips. Lint, type checks, packaging, Bedrock, and current required checks also passed. Source review plus existing hosted CI only; no local test execution or live-API validation.

@jbeckwith-oai jbeckwith-oai changed the title fix(examples): validate stream resume prerequisites examples: validate stream resume prerequisites Aug 24, 2026
@jbeckwith-oai
jbeckwith-oai added this pull request to the merge queue Aug 24, 2026
Merged via the queue into main with commit 1e7b3da Aug 24, 2026
23 checks passed
@jbeckwith-oai
jbeckwith-oai deleted the codex/streaming-previous-response-prerequisites branch August 24, 2026 16:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

codex-maintenance Low-risk maintenance changes created by Codex

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants