Skip to content

Pull requests: openai/evals

New pull request New
130 Open 1,272 Closed
130 Open 1,272 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Assigned to nobody Loading
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

Allow valid_answers without extra completion options
#1760 opened Aug 21, 2026 by sylvesterkaczmarek Loading…
1
Return no consensus instead of crashing SelfConsistencySolver
#1758 opened Aug 21, 2026 by sylvesterkaczmarek Loading…
1
Forward runtime kwargs through FewShotSolver
#1756 opened Aug 21, 2026 by sylvesterkaczmarek Loading…
1
Use scheduled sample count for async progress
#1754 opened Aug 21, 2026 by sylvesterkaczmarek Loading…
1
Make the MAFAND example Windows-safe and UTF-8 explicit
#1752 opened Aug 21, 2026 by sylvesterkaczmarek Loading…
1
Parse EVALS_SHOW_EVAL_PROGRESS as a boolean
#1751 opened Aug 21, 2026 by sylvesterkaczmarek Loading…
1
Use Ruff instead of the duplicate Autoflake hook
#1749 opened Aug 21, 2026 by sylvesterkaczmarek Loading…
1
Validate Match few-shot rows before evaluation
#1748 opened Aug 21, 2026 by sylvesterkaczmarek Loading…
1
Enforce pre-commit hooks on pull request changes
#1747 opened Aug 21, 2026 by sylvesterkaczmarek Loading…
1
1
Forward typed completion args to OpenAI requests
#1746 opened Aug 21, 2026 by sylvesterkaczmarek Loading…
2
Add seeded random few-shot selection to Match
#1744 opened Aug 21, 2026 by sylvesterkaczmarek Loading…
1
Add built-in scoring for precomputed outputs
#1743 opened Aug 21, 2026 by sylvesterkaczmarek Loading…
1
Harden routing for modern OpenAI text models
#1742 opened Aug 21, 2026 by sylvesterkaczmarek Loading…
1
Restore recorder context after exceptions
#1741 opened Aug 21, 2026 by sylvesterkaczmarek Loading…
1
Keep failed eval-set commands out of resume checkpoints
#1740 opened Aug 21, 2026 by sylvesterkaczmarek Loading…
1
Add explain mode to HumanCliSolver
#1737 opened Aug 21, 2026 by sylvesterkaczmarek Loading…
1
Add strict ExactMatch eval template
#1736 opened Aug 21, 2026 by sylvesterkaczmarek Loading…
1
Add --version to oaieval
#1735 opened Aug 21, 2026 by sylvesterkaczmarek Loading…
1
Clarify JSONL field mapping for eval datasets
#1734 opened Aug 21, 2026 by sylvesterkaczmarek Loading…
1
Retry API failures by exception instead of sentinel
#1733 opened Aug 21, 2026 by sylvesterkaczmarek Loading…
1
Document the pipx package name
#1732 opened Aug 21, 2026 by sylvesterkaczmarek Loading…
1
Expose the active eval run ID
#1731 opened Aug 21, 2026 by sylvesterkaczmarek Loading…
1
Preflight known context-window overflows
#1730 opened Aug 21, 2026 by sylvesterkaczmarek Loading…
1
Reset max_samples state for every oaieval run
#1728 opened Aug 21, 2026 by sylvesterkaczmarek Loading…
1
Avoid mutating chat prompts during formatting scrub
#1726 opened Aug 21, 2026 by sylvesterkaczmarek Loading…
1
ProTip! Mix and match filters to narrow down what you’re looking for.