examples: validate parsed chat completion events - #517
Conversation
jbeckwith-oai
added
the
codex-maintenance
Castiron custom code✅ No new custom-code files detected. 48 mixed files remain; 0 existing customizations changed. Compared 48 existing customizations unchanged
8 more in the full report. A changed generated baseline means this report cannot reliably identify which handwritten lines changed. Inspect the custom-code diffDownload the exact patch produced by this run (requires repository access): gh run download 32662590766 --repo openai/openai-ruby \
--name castiron-custom-code-32662590766-1 --dir /tmp/castiron-custom-code-32662590766-1
git apply --stat /tmp/castiron-custom-code-32662590766-1/custom-code.patch
cat /tmp/castiron-custom-code-32662590766-1/custom-code.patchOr reproduce it from an SDK checkout containing the vendored reporter: git fetch --no-tags origin edfb30b2973d68b4a19ce3f86f1de7313f730702 c6d841f36e78e318841df10735dce2e187ed310e
python3 scripts/castiron/custom_code_report.py report \
--base edfb30b2973d68b4a19ce3f86f1de7313f730702 \
--head c6d841f36e78e318841df10735dce2e187ed310e --fetch --require-head-hash --public \
--out /tmp/castiron-custom-code-c6d841f36e78
cat /tmp/castiron-custom-code-c6d841f36e78/custom-code.patchThis is the current full custom patch for mixed files, not an attribution of only the handwritten lines changed by this PR. |
jbeckwith-oai
changed the title
84fecdc to
c6d841f
Compare
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
apcha-oai
left a comment
apcha-oai
left a comment
There was a problem hiding this comment.
[sdk codex review] Approved at c6d841f36e78e318841df10735dce2e187ed310e after two independent, full code reviews and reconciliation of existing feedback. No actionable findings.
The example preserves refusal handling and valid-event ordering while rejecting runs with no parsed CalendarEvent. All eight regression/characterization cases were examined, including empty, refused, wrong-type, and mixed choices. The contributor-guidance change was also reviewed; no public SDK parsing or API behavior changes.
Existing hosted CI for the reviewed head merged with edfb30b2973d68b4a19ce3f86f1de7313f730702 passed on Ruby 3.3, 3.4, and 4.0: 1,199 tests and 10,643 assertions per version, with no failures, errors, or skips. Lint, type checks, packaging, Bedrock, and current required checks also passed. Source review plus existing hosted CI only; no local test execution or live-API validation.
2 participants
Summary
CalendarEventinstances from the handwritten structured-output Chat Completions example, preserving existing successful output and refusal handling.The response did not contain a parsed CalendarEventwhen choices are empty, all refused, or contain only nil/wrong-type parsed values.examples:as the required Conventional Commit prefix for example-focused commits and pull request titles.Evidence and scope
The new regression suite failed 7 of 8 tests against the unchanged example: empty choices and refused/invalid values exited successfully, while mixed invalid values were printed. This change is limited to one handwritten, generator-independent example, its dedicated new test, and a concise
AGENTS.mdcontributor-guidance clarification. It does not modify generated SDK code, response parsing, model behavior, transport, authentication, serialization, public APIs, dependencies, or CI.Validation
mise exec ruby@3.3.12 -- ./scripts/test— 1,199 tests, 10,642 assertions, 0 failures, 1 existing skip.mise exec ruby@3.4.10 -- ./scripts/test— 1,199 tests, 10,642 assertions, 0 failures, 1 existing skip.mise exec ruby@4.0.6 -- ./scripts/test— 1,199 tests, 10,642 assertions, 0 failures, 1 existing skip.TEST='test/openai/structured_outputs*_test.rb' mise exec ruby@4.0.6 -- bundle exec rake test— 11 tests, 66 assertions.mise exec ruby@4.0.6 -- bundle exec rake lint— RuboCop, rubyfmt, directive validation, Sorbet, and 1,239 RBS files pass.python3 -m unittest discover -s scripts/castiron -p 'test_custom_code*.py'— 51 tests pass, 1 existing skip.python3 -B -I scripts/castiron/custom_code_budget.py check --repo . --base edfb30b2973d68b4a19ce3f86f1de7313f730702 --head c6d841f36e78e318841df10735dce2e187ed310e --public --out /private/tmp/openai-ruby-calendar-event-custom-budget-v2— 2,659 / 4,000 custom lines; budget and isolation pass.bundle exec rake test:examples:inventory,git diff --check, pinned-base scope review, extensive general review, and thermo-nuclear maintainability review all pass.Local runs used the existing cached bundle for each Ruby version and
TMPDIR=/private/tmpto avoid the pre-existing macOS/varversus/private/varpath-alias issue in formatter tests.