Skip to content

[#22733][fix] Kimi-K2.5 reasoning parser: keep reasoning cut off by the token limit out of the content - #22734

Open
LeizhenZhang wants to merge 1 commit into
NVIDIA:mainfrom
LeizhenZhang:fix/kimi-k25-truncated-reasoning
Open

LeizhenZhang wants to merge 1 commit into
NVIDIA:mainfrom
LeizhenZhang:fix/kimi-k25-truncated-reasoning

Conversation

@LeizhenZhang

@LeizhenZhang LeizhenZhang commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

Description

Fixes #22733.

In thinking mode the Kimi-K2.5 chat template prefills <think>. When max_tokens stops the model before </think>, the output is reasoning only. KimiK2ReasoningParser.parse returned it as content, while parse_delta and finish stream it as reasoning_content. So a non-streaming request showed the chain of thought as the answer, and the same output streamed split correctly.

The branch that returned it as content was meant for instant mode. Instant mode is thinking=False, and __init__ then sets reasoning_at_start=False, so an output without <think> is content through the early return at the top of parse. The branch is reached only by truncated reasoning, and now returns it as reasoning. DeepSeekR1Parser.parse treats the same case the same way.

Kimi-K2.5, thinking=True, output stopped before </think> (73 characters) content reasoning_content
parse() before 73 characters empty
parse() after empty 73 characters
streaming, before and after empty 73 characters

kimi_k2 (reasoning_at_start=False) and thinking=False keep their behavior.

Test Coverage

tests/unittest/llmapi/test_reasoning_parser.py

  • test_kimi_k25_parse_matches_streaming. Four outputs (two truncated, two complete controls) with no kwargs and with thinking=True, parse() against parse_delta over 3-character deltas plus finish().
  • test_kimi_k25_truncated_reasoning_is_reasoning.
  • test_kimi_k25_instant_mode_output_is_content, for thinking=False.

Results on a CPU host with the parser sources loaded from the checkout (no compiled bindings).

Run Result
test_reasoning_parser.py on main 358 passed, 1 skipped
this PR's test file on main 5 failed (the 4 truncated cases and test_kimi_k25_truncated_reasoning_is_reasoning), 363 passed, 1 skipped
this PR's test file on this branch 368 passed, 1 skipped

Not tested: a served Kimi-K2.5 model end to end, and the Responses API path.

PR Checklist

  • PR description explains what and why.
  • Follows the coding guidelines.
  • Test cases cover the change.
  • The test file is already in the CPU test list.

Dev Engineer Review

  • KimiK2ReasoningParser.parse now returns unclosed output as reasoning_content and sets content to empty. It no longer checks reasoning_at_start on this branch. Non-streaming results for truncated reasoning now match streaming.
  • Risk: the change assumes that only truncated reasoning reaches this branch. Confirm that thinking=False sets reasoning_at_start=False in every path, so instant-mode output stays content.
  • Not tested: a served Kimi-K2.5 model end to end, and the Responses API path.

QA Engineer Review

Changed test file: tests/unittest/llmapi/test_reasoning_parser.py (+59 lines). Added tests:

  • test_kimi_k25_parse_matches_streaming
  • test_kimi_k25_truncated_reasoning_is_reasoning
  • test_kimi_k25_instant_mode_output_is_content

Coverage:

  • Whole-text parse() matches chunked parse_delta() plus finish(), with default options and with thinking=True.
  • Truncated reasoning returns as reasoning_content.
  • thinking=False output returns as content.

Test-db (CI) or qa list membership: not available.

Coverage verdict: needs follow-up. Served-model and Responses API paths are not covered.

Per-File QA Perspective

  • tensorrt_llm/llmapi/reasoning_parser.py: Verify that non-streaming truncated Kimi K2.5 thinking output returns reasoning_content with empty content. Verify that thinking=False output still returns content.
  • tests/unittest/llmapi/test_reasoning_parser.py: Covers the three Kimi K2.5 cases above. List membership in test-db/ or qa/ is not available.

… token limit out of the content

In thinking mode the Kimi-K2.5 template prefills <think>, so an output
with no </think> and no tool-call section is reasoning that the token
limit stopped. parse() returned it as content while parse_delta() and
finish() stream it as reasoning_content, so the same output split
differently between streaming and non-streaming requests. Instant mode
(thinking=False) is unaffected, because the parser then starts outside
the reasoning block.

Signed-off-by: LeizhenZhang <zhangleizhen645@gmail.com>
@coderabbitai

coderabbitai Bot commented Nov 21, 2026

Copy link
Copy Markdown
Contributor

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

When Kimi K2.5 output has no closing reasoning marker or tool-call section, the parser assigns the remaining text to reasoning_content. Tests cover thinking options and compare whole-text parsing with streaming output.

Changes

Kimi K2.5 reasoning parsing

Layer / File(s) Summary
Unterminated reasoning classification
tensorrt_llm/llmapi/reasoning_parser.py, tests/unittest/llmapi/test_reasoning_parser.py
When neither a closing reasoning marker nor a tool-call section ends the block, the parser assigns the remaining text to reasoning. Tests cover thinking=True, thinking=False, and whole-parse versus streaming parity.

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix · Severity of issue fixed: Medium

Suggested reviewers: junyixu-nv, tongyuantongyu, zhaoyangwang-nvidia

Merge Risk: Minimal · up to 21629

This change makes non-streaming Kimi-K2.5 output cut off before the closing think marker return as reasoning content, matching streaming behavior. Instant mode is unchanged. Tests cover both modes and streaming parity. No merge-blocking risk was found.

Security architecture risk: Minimal · up to 21629

The fix keeps incomplete reasoning separate from answer content without expanding access, privileges, or execution capabilities. No introduced or worsened architecture-level security risk was identified.

Pre-merge checks: Passed 4

Check name Status Explanation
Title check Passed The title names the main change: the Kimi-K2.5 reasoning parser now keeps reasoning cut off by the token limit out of the content field. It also carries the ticket and type tag from the repository tem…
Description check Passed The description follows the repository template closely. It has a Description section that explains the cause (the parse branch returned truncated reasoning as content while streaming returned it as r…
Linked Issues check Passed Issue #22733 requires that Kimi-K2.5 output cut off before </think> in thinking mode be returned as reasoning_content, with content empty, in the non-streaming parse() path, matching the strea…
Out of Scope Changes check Passed The whole-PR diff against merge base 97e0614 touches only tensorrt_llm/llmapi/reasoning_parser.py (the no-end-marker branch of KimiK2ReasoningParser.parse, +7/-13) and the tests in `tests/unitt…
Full details: Title check

Explanation

The title names the main change: the Kimi-K2.5 reasoning parser now keeps reasoning cut off by the token limit out of the content field. It also carries the ticket and type tag from the repository template. This matches the raw summary, which says unclosed output in the thinking branch is now returned as reasoning_content, and the PR objectives, which describe the same fix for linked issue #22733.

Full details: Description check

Explanation

The description follows the repository template closely. It has a Description section that explains the cause (the parse branch returned truncated reasoning as content while streaming returned it as reasoning) and the fix. It has a Test Coverage section that lists three named tests and gives run results. It has a PR Checklist with the items marked. It covers the behavior for thinking=False and kimi_k2, and it notes what was not tested. The description does not restate the template's title-format guidance, which is not needed in the body.

Full details: Linked Issues check

Explanation

Issue #22733 requires that Kimi-K2.5 output cut off before &lt;/think&gt; in thinking mode be returned as reasoning_content, with content empty, in the non-streaming parse() path, matching the streaming parse_delta() plus finish() path. The head diff of KimiK2ReasoningParser.parse (tensorrt_llm/llmapi/reasoning_parser.py) replaces the reasoning_at_start-dependent no-end-marker branch with an unconditional assignment: reasoning_content = text and content = "". This directly implements the requirement. The new tests test_kimi_k25_parse_matches_streaming and test_kimi_k25_truncated_reasoning_is_reasoning cover the truncated case for parse() and streaming parity, and test_kimi_k25_instant_mode_output_is_content covers the thinking=False case that the issue does not ask to change. The PR reports the test results on a CPU host; I did not re-run them. The instant-mode path depends on the unchanged early return in parse and the unchanged __init__ setting of reasoning_at_start=False; the PR description states both, and the diff does not alter them. Non-coding items are not applicable.

Full details: Out of Scope Changes check

Explanation

The whole-PR diff against merge base 97e0614 touches only tensorrt_llm/llmapi/reasoning_parser.py (the no-end-marker branch of KimiK2ReasoningParser.parse, +7/-13) and the tests in tests/unittest/llmapi/test_reasoning_parser.py, which the summary lists as +59 lines of Kimi K2.5 tests. The parser change is the fix the issue requests. The tests support that fix. The PR description states that kimi_k2 and thinking=False behavior are unchanged, and the diff does not modify those code paths. No unrelated change was found.

Finishing Touches

Generate unit tests (beta)

  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Recent review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between 260e6b8 and 216292e.

Files selected for processing: 2

  • tensorrt_llm/llmapi/reasoning_parser.py
  • tests/unittest/llmapi/test_reasoning_parser.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Kimi-K2.5 reasoning parser returns reasoning cut off by max_tokens as content in non-streaming responses

1 participant