You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[#22733][fix] Kimi-K2.5 reasoning parser: keep reasoning cut off by the token limit out of the content - #22734
In thinking mode the Kimi-K2.5 chat template prefills <think>. When max_tokens stops the model before </think>, the output is reasoning only. KimiK2ReasoningParser.parse returned it as content, while parse_delta and finish stream it as reasoning_content. So a non-streaming request showed the chain of thought as the answer, and the same output streamed split correctly.
The branch that returned it as content was meant for instant mode. Instant mode is thinking=False, and __init__ then sets reasoning_at_start=False, so an output without <think> is content through the early return at the top of parse. The branch is reached only by truncated reasoning, and now returns it as reasoning. DeepSeekR1Parser.parse treats the same case the same way.
Kimi-K2.5, thinking=True, output stopped before </think> (73 characters)
content
reasoning_content
parse() before
73 characters
empty
parse() after
empty
73 characters
streaming, before and after
empty
73 characters
kimi_k2 (reasoning_at_start=False) and thinking=False keep their behavior.
Test Coverage
tests/unittest/llmapi/test_reasoning_parser.py
test_kimi_k25_parse_matches_streaming. Four outputs (two truncated, two complete controls) with no kwargs and with thinking=True, parse() against parse_delta over 3-character deltas plus finish().
test_kimi_k25_truncated_reasoning_is_reasoning.
test_kimi_k25_instant_mode_output_is_content, for thinking=False.
Results on a CPU host with the parser sources loaded from the checkout (no compiled bindings).
Run
Result
test_reasoning_parser.py on main
358 passed, 1 skipped
this PR's test file on main
5 failed (the 4 truncated cases and test_kimi_k25_truncated_reasoning_is_reasoning), 363 passed, 1 skipped
this PR's test file on this branch
368 passed, 1 skipped
Not tested: a served Kimi-K2.5 model end to end, and the Responses API path.
PR Checklist
PR description explains what and why.
Follows the coding guidelines.
Test cases cover the change.
The test file is already in the CPU test list.
Dev Engineer Review
KimiK2ReasoningParser.parse now returns unclosed output as reasoning_content and sets content to empty. It no longer checks reasoning_at_start on this branch. Non-streaming results for truncated reasoning now match streaming.
Risk: the change assumes that only truncated reasoning reaches this branch. Confirm that thinking=False sets reasoning_at_start=False in every path, so instant-mode output stays content.
Not tested: a served Kimi-K2.5 model end to end, and the Responses API path.
QA Engineer Review
Changed test file: tests/unittest/llmapi/test_reasoning_parser.py (+59 lines). Added tests:
test_kimi_k25_parse_matches_streaming
test_kimi_k25_truncated_reasoning_is_reasoning
test_kimi_k25_instant_mode_output_is_content
Coverage:
Whole-text parse() matches chunked parse_delta() plus finish(), with default options and with thinking=True.
Truncated reasoning returns as reasoning_content.
thinking=False output returns as content.
Test-db (CI) or qa list membership: not available.
Coverage verdict: needs follow-up. Served-model and Responses API paths are not covered.
Per-File QA Perspective
tensorrt_llm/llmapi/reasoning_parser.py: Verify that non-streaming truncated Kimi K2.5 thinking output returns reasoning_content with empty content. Verify that thinking=False output still returns content.
tests/unittest/llmapi/test_reasoning_parser.py: Covers the three Kimi K2.5 cases above. List membership in test-db/ or qa/ is not available.
… token limit out of the content
In thinking mode the Kimi-K2.5 template prefills <think>, so an output
with no </think> and no tool-call section is reasoning that the token
limit stopped. parse() returned it as content while parse_delta() and
finish() stream it as reasoning_content, so the same output split
differently between streaming and non-streaming requests. Instant mode
(thinking=False) is unaffected, because the parser then starts outside
the reasoning block.
Signed-off-by: LeizhenZhang <zhangleizhen645@gmail.com>
No actionable comments were generated in the recent review.
Walkthrough
When Kimi K2.5 output has no closing reasoning marker or tool-call section, the parser assigns the remaining text to reasoning_content. Tests cover thinking options and compare whole-text parsing with streaming output.
When neither a closing reasoning marker nor a tool-call section ends the block, the parser assigns the remaining text to reasoning. Tests cover thinking=True, thinking=False, and whole-parse versus streaming parity.
This change makes non-streaming Kimi-K2.5 output cut off before the closing think marker return as reasoning content, matching streaming behavior. Instant mode is unchanged. Tests cover both modes and streaming parity. No merge-blocking risk was found.
Security architecture risk: Minimal · up to 21629
The fix keeps incomplete reasoning separate from answer content without expanding access, privileges, or execution capabilities. No introduced or worsened architecture-level security risk was identified.
Pre-merge checks: 4
Check name
Status
Explanation
Title check
The title names the main change: the Kimi-K2.5 reasoning parser now keeps reasoning cut off by the token limit out of the content field. It also carries the ticket and type tag from the repository tem…
Description check
The description follows the repository template closely. It has a Description section that explains the cause (the parse branch returned truncated reasoning as content while streaming returned it as r…
Linked Issues check
Issue #22733 requires that Kimi-K2.5 output cut off before </think> in thinking mode be returned as reasoning_content, with content empty, in the non-streaming parse() path, matching the strea…
Out of Scope Changes check
The whole-PR diff against merge base 97e0614 touches only tensorrt_llm/llmapi/reasoning_parser.py (the no-end-marker branch of KimiK2ReasoningParser.parse, +7/-13) and the tests in `tests/unitt…
Full details: Title check
Explanation
The title names the main change: the Kimi-K2.5 reasoning parser now keeps reasoning cut off by the token limit out of the content field. It also carries the ticket and type tag from the repository template. This matches the raw summary, which says unclosed output in the thinking branch is now returned as reasoning_content, and the PR objectives, which describe the same fix for linked issue #22733.
Full details: Description check
Explanation
The description follows the repository template closely. It has a Description section that explains the cause (the parse branch returned truncated reasoning as content while streaming returned it as reasoning) and the fix. It has a Test Coverage section that lists three named tests and gives run results. It has a PR Checklist with the items marked. It covers the behavior for thinking=False and kimi_k2, and it notes what was not tested. The description does not restate the template's title-format guidance, which is not needed in the body.
Full details: Linked Issues check
Explanation
Issue #22733 requires that Kimi-K2.5 output cut off before </think> in thinking mode be returned as reasoning_content, with content empty, in the non-streaming parse() path, matching the streaming parse_delta() plus finish() path. The head diff of KimiK2ReasoningParser.parse (tensorrt_llm/llmapi/reasoning_parser.py) replaces the reasoning_at_start-dependent no-end-marker branch with an unconditional assignment: reasoning_content = text and content = "". This directly implements the requirement. The new tests test_kimi_k25_parse_matches_streaming and test_kimi_k25_truncated_reasoning_is_reasoning cover the truncated case for parse() and streaming parity, and test_kimi_k25_instant_mode_output_is_content covers the thinking=False case that the issue does not ask to change. The PR reports the test results on a CPU host; I did not re-run them. The instant-mode path depends on the unchanged early return in parse and the unchanged __init__ setting of reasoning_at_start=False; the PR description states both, and the diff does not alter them. Non-coding items are not applicable.
Full details: Out of Scope Changes check
Explanation
The whole-PR diff against merge base 97e0614 touches only tensorrt_llm/llmapi/reasoning_parser.py (the no-end-marker branch of KimiK2ReasoningParser.parse, +7/-13) and the tests in tests/unittest/llmapi/test_reasoning_parser.py, which the summary lists as +59 lines of Kimi K2.5 tests. The parser change is the fix the issue requests. The tests support that fix. The PR description states that kimi_k2 and thinking=False behavior are unchanged, and the diff does not modify those code paths. No unrelated change was found.
Finishing Touches
Generate unit tests (beta)
Create a new PR
Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Reviewing files that changed from the base of the PR and between 260e6b8 and 216292e.
Files selected for processing: 2
tensorrt_llm/llmapi/reasoning_parser.py
tests/unittest/llmapi/test_reasoning_parser.py
Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.
This branch has not been deployed
No deployments
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixes #22733.
In thinking mode the Kimi-K2.5 chat template prefills
<think>. Whenmax_tokensstops the model before</think>, the output is reasoning only.KimiK2ReasoningParser.parsereturned it ascontent, whileparse_deltaandfinishstream it asreasoning_content. So a non-streaming request showed the chain of thought as the answer, and the same output streamed split correctly.The branch that returned it as content was meant for instant mode. Instant mode is
thinking=False, and__init__then setsreasoning_at_start=False, so an output without<think>is content through the early return at the top ofparse. The branch is reached only by truncated reasoning, and now returns it as reasoning.DeepSeekR1Parser.parsetreats the same case the same way.thinking=True, output stopped before</think>(73 characters)contentreasoning_contentparse()beforeparse()afterkimi_k2(reasoning_at_start=False) andthinking=Falsekeep their behavior.Test Coverage
tests/unittest/llmapi/test_reasoning_parser.pytest_kimi_k25_parse_matches_streaming. Four outputs (two truncated, two complete controls) with no kwargs and withthinking=True,parse()againstparse_deltaover 3-character deltas plusfinish().test_kimi_k25_truncated_reasoning_is_reasoning.test_kimi_k25_instant_mode_output_is_content, forthinking=False.Results on a CPU host with the parser sources loaded from the checkout (no compiled bindings).
test_reasoning_parser.pyonmainmaintest_kimi_k25_truncated_reasoning_is_reasoning), 363 passed, 1 skippedNot tested: a served Kimi-K2.5 model end to end, and the Responses API path.
PR Checklist
Dev Engineer Review
KimiK2ReasoningParser.parsenow returns unclosed output asreasoning_contentand setscontentto empty. It no longer checksreasoning_at_starton this branch. Non-streaming results for truncated reasoning now match streaming.thinking=Falsesetsreasoning_at_start=Falsein every path, so instant-mode output stays content.QA Engineer Review
Changed test file:
tests/unittest/llmapi/test_reasoning_parser.py(+59 lines). Added tests:test_kimi_k25_parse_matches_streamingtest_kimi_k25_truncated_reasoning_is_reasoningtest_kimi_k25_instant_mode_output_is_contentCoverage:
parse()matches chunkedparse_delta()plusfinish(), with default options and withthinking=True.reasoning_content.thinking=Falseoutput returns ascontent.Test-db (CI) or qa list membership: not available.
Coverage verdict: needs follow-up. Served-model and Responses API paths are not covered.
Per-File QA Perspective
tensorrt_llm/llmapi/reasoning_parser.py: Verify that non-streaming truncated Kimi K2.5 thinking output returnsreasoning_contentwith emptycontent. Verify thatthinking=Falseoutput still returnscontent.tests/unittest/llmapi/test_reasoning_parser.py: Covers the three Kimi K2.5 cases above. List membership intest-db/orqa/is not available.