Skip to content

fix: filter truncated candidates from batched best-of-n sampling - #342

Merged
codelion merged 1 commit into
algorithmicsuperintelligence:mainfrom
liuzhengyang699:fix/bon-truncated-candidates
Oct 10, 2026
Merged

codelion merged 1 commit into
algorithmicsuperintelligence:mainfrom
liuzhengyang699:fix/bon-truncated-candidates

Conversation

@liuzhengyang699

Copy link
Copy Markdown
Contributor

The sequential fallback skips length-truncated completions, but the batched n path accepts them. A judge can then rank an incomplete answer above the complete candidates. Whitespace-only responses can also reach the judge.

Apply the same validity rule to both generation paths: keep nonblank content only when the finish reason is not length. An entirely invalid batch uses the existing sequential fallback.

Validation: 9 tests passed across tests/test_bon_candidate_filtering.py and tests/test_approaches.py. Three new cases fail before the fix; they verify the selected answer, which candidates reach the judge, fallback behavior, and token accounting for successful responses. Ruff checks on the new test and git diff --check passed.

@CLAassistant

CLAassistant commented Oct 10, 2026 •

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@codelion
codelion merged commit dabd59d into algorithmicsuperintelligence:main Oct 10, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants