Repository navigation
[#22735][fix] Kimi K2 tool parser: skip a call to an undeclared function in streaming too - #22736
LeizhenZhang wants to merge 1 commit into
Conversation
…n in streaming too detect_and_parse skips a call to a function the request does not declare, as test_undefined_tool expects. Streaming reported the same call, so a streamed response carried it with finish_reason "tool_calls" while the non-streaming response had no call. Streaming now holds such a call until its end token and drops it without giving it a tool index. Signed-off-by: LeizhenZhang <zhangleizhen645@gmail.com>
|
No actionable comments were generated in the recent review. WalkthroughThe Kimi K2 streaming parser now discards undeclared function calls after their call-end token. Regression tests cover multiple chunk sizes and an undeclared call between declared calls. ChangesKimi K2 streaming parser
Estimated code review effort: 2 (Simple) | ~10 minutes Change: Bug fix · Severity of issue fixed: Low Merge Risk:
|
Description
With
--tool_parser kimi_k2, a call to a function the request does not declare is skipped bydetect_and_parsebut streamed byparse_streaming_increment. A streamed response carries the call and ends withfinish_reason: "tool_calls". The same output without streaming has no call and ends withfinish_reason: "stop".detect_and_parseskips such a call on purpose (continueafter the "undefined function" warning), andTestKimiK2ToolParser::test_undefined_toolexpects it. This PR applies the same rule to streaming. In_parse_buffered_tool_call, a call whose function is not in the request's tools is held until its<|tool_call_end|>arrives and then dropped with the same warning. It gets no tool index, so the calls after it keep consecutive indices, and nothing is added toprev_tool_call_arrorstreamed_args_for_tool.The other tool parsers on
mainreport such a call in both paths. If you prefer that Kimi K2 report it too, the fix is to remove thecontinueindetect_and_parseand updatetest_undefined_tool. I kept the behavior the existing test documents.Fixes #22735.
Before and after
mainundefined_funcdetect_and_parseundefined_funcstreamedget_weather,undefined_func,search_webdetect_and_parseget_weather,search_webget_weather,search_webtest_streamed_calls_match_whole_text_parseget_weather,undefined_func,search_webget_weather,search_web(indices 0 and 1)Test Coverage
TestKimiK2ToolParser::test_undefined_tool_is_not_streamed(three chunk sizes).undeclared_call_between_callsin_KIMI_K2_OUTPUTS, used byTestKimiK2StreamingMatchesWholeTextParsefor the calls (whole output, one character, fixed sizes, every two-way split) and for the streamed text.-k KimiK2intests/unittest/llmapi/apps/test_tool_parsers.py: 4 failed and 41 passed withmain's parser (the new cases), 45 passed with this PR.test_tool_parsers.py: 1160 passed and 48 failed with this PR. The 48 are the same tests that fail onmainin my local setup, which runs the parsers from this checkout on the 1.4.0rc1 wheel. None of them is a Kimi K2 test.pre-commiton the changed files passes.Not tested: a live server. The parser was driven directly.
PR Checklist
Dev Engineer Review
KimiK2ToolParser._parse_buffered_tool_callnow drops calls to undeclared functions in streaming. This matches the existing non-streaming behavior andtest_undefined_tool.<|tool_call_end|>. Its text is not emitted during streaming, and the call gets no index or streaming state.QA Engineer Review
tests/unittest/llmapi/apps/test_tool_parsers.py(+22 lines).TestKimiK2ToolParser::test_undefined_tool_is_not_streamed(chunk sizes 1, 7, 1000).undeclared_call_between_callscase toTestKimiK2StreamingMatchesWholeTextParse.test-db/orqa/: not available.main. Not verified.Per-file QA perspective
kimi_k2_tool_parser.py: Streamed responses no longer include undeclared function calls.finish_reasonshould now match non-streamed output for the same text.test_tool_parsers.py: Adds regression coverage for the streaming and whole-text parity of undeclared calls.