Skip to content

[https://nvbugs/7125734][fix] Read early responses from the response channel - #22698

Merged
trtllm-agent merged 2 commits into
NVIDIA:mainfrom
tensorrt-cicd:repair-bot-v2/nvbug-7125734-r567
Nov 21, 2026
Merged

trtllm-agent merged 2 commits into
NVIDIA:mainfrom
tensorrt-cicd:repair-bot-v2/nvbug-7125734-r567

Conversation

@trtllm-agent

@trtllm-agent trtllm-agent commented Nov 20, 2026 •

Copy link
Copy Markdown
Collaborator

Description

The _first_token_executor fixture in tests/unittest/_torch/pyexecutor/test_py_executor.py initializes a ResponseChannel following #21904. The early-response regression added in #22346 still reads executor.responses, although the response-delivery path stores results in executor._response_channel.responses (tensorrt_llm/_torch/pyexecutor/request_delivery.py).

Read the channel's queue in the seven existing response lookups. Keep all eight assertions and the termination checks unchanged, and remove only this case's waiver from tests/integration/test_lists/waives.txt.

Test Coverage

  • Existing regression: unittest/_torch/pyexecutor/test_py_executor.py::test_early_response_suppresses_repeat_and_leaves_terminal_for_normal_pass.
  • Existing CPU CI registration: unittest/_torch/pyexecutor in tests/integration/test_lists/test-db/l0_cpu.yml.
  • ARM64 CPU, zero GPUs: run the complete unittest/_torch/pyexecutor entry through tests/integration/defs/test_unittests.py with --test-list and --unittest-markexpr=cpu_only, keeping its inner tests together in one pytest process.
  • Before, on main snapshot 1d3b8a7cde9f6af93ff7b80916150a60e47d0732 with only this case unwaived: 5,910 passed, 22 skipped, 1 failed. The exact regression raises AttributeError: 'PyExecutor' object has no attribute 'responses'.
  • After this change, with the same batch and environment: 5,911 passed, 22 skipped, 0 failed. The exact regression passes unskipped. Neighboring first-token, prefill-logit and logprob tests pass in both runs.
  • ruff check and ruff format --check pass for tests/unittest/_torch/pyexecutor/test_py_executor.py.

Fixes https://nvbugs/7125734

Dev Engineer Review

  • Test-only change. No source files changed.
  • Seven response lookups in test_early_response_suppresses_repeat_and_leaves_terminal_for_normal_pass now read executor._response_channel.responses instead of executor.responses. Assertions and termination checks are unchanged.
  • The test now depends on the private _response_channel attribute. A refactor of that attribute breaks the test.
  • The nvbugs/7125734 waiver is removed, so this test runs unskipped.

QA Engineer Review

  • Modified: tests/unittest/_torch/pyexecutor/test_py_executor.py (+7/-7).
  • Modified: tests/integration/test_lists/waives.txt (one SKIP entry removed).
  • Covers early-response suppression of repeat responses and single terminal emission.
  • Listing in test-db/ or qa/ was not verified.
  • Author reports a local ARM64 CPU run with 0 failures. This is not a completed CI check.
  • Coverage verdict: needs follow-up. Confirm the test passes in the intended CI list now that the waiver is gone.

Per-File QA Perspective

  • test_py_executor.py: Checks early responses through the response channel. QA should confirm the test passes in CI.
  • waives.txt: Removing the waiver makes the test run in CI. QA should watch for new failures.

…channel

In test_py_executor.py, _first_token_executor creates a ResponseChannel
(NVIDIA#21904), but the early-response regression added in NVIDIA#22346 still reads
executor.responses.

Read the channel's response queue in the seven existing lookups, keeping
the assertions and termination checks unchanged. Remove the matching
waiver from tests/integration/test_lists/waives.txt.

Signed-off-by: trtllm-agent <296075020+trtllm-agent@users.noreply.github.com>
@coderabbitai

coderabbitai Bot commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

The first-token response test now reads emitted responses from executor._response_channel.responses. Its response and termination expectations are unchanged. The integration waiver for the test was removed.

Changes

PyExecutor response test

Layer / File(s) Summary
Update response test and remove waiver
tests/unittest/_torch/pyexecutor/test_py_executor.py, tests/integration/test_lists/waives.txt
The test reads response counts and results from _response_channel.responses; its response and termination expectations remain unchanged. The integration waiver was removed.

Estimated code review effort: 1 (Trivial) | ~5 minutes

Change: Other

Merge Risk: Minimal · up to 08c92

This is a test-only fix that points an existing regression at the response channel and re-enables it by removing its waiver. It does not change runtime behavior, and no merge-blocking risk was found.

Security architecture risk: Minimal · up to d89c7

The change restores an existing regression test without changing its expected behavior or production execution paths. No introduced or worsened architecture-level security risk was identified.

Pre-merge checks: Passed 4

Check name Status Explanation
Title check Passed The title names the NVBug ID and the [fix] type, and it describes the main change: early responses are now read from the response channel. It follows the repository template. The title is specific and…
Description check Passed The description covers the problem, the fix, and test coverage. It names the affected fixture, the file and line references, the regression test, the CI registration, and before and after results. It …
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title names the NVBug ID and the [fix] type, and it describes the main change: early responses are now read from the response channel. It follows the repository template. The title is specific and matches the changeset, which updates seven response lookups in the early-response test and removes one waiver.

Full details: Description check

Explanation

The description covers the problem, the fix, and test coverage. It names the affected fixture, the file and line references, the regression test, the CI registration, and before and after results. It also explains the waiver removal and links the NVBug. The description does not follow the template's '## PR Checklist' section, and it omits the template's 'Description' explanation style of a short issue and solution, but it includes the required information. The checklist is unticked, which is acceptable for a mostly complete description.

Finishing Touches

Generate unit tests (beta)

  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Recent review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between d89c7fe and 08c9269.

Files selected for processing: 1

  • tests/integration/test_lists/waives.txt
Files with no reviewable changes: 1

  • tests/integration/test_lists/waives.txt

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

@trtllm-agent-review trtllm-agent-review left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR. It points the early first-token response regression test at the executor's ResponseChannel, where delivered responses have lived since #21904, and removes the test's nvbugs/7125734 waiver.

Reviewed at commit d89c7fe9d1 (2 files, +7/−8).

Approve: no Blocker found.

What this review checked
  • At the base the test fails with AttributeError, because PyExecutor no longer has responses. The new lookups read the dict that _gather_and_publish_responses delivers into (py_executor.py:13409, request_delivery.py:83-86), and no other test or production code still reads the moved attributes.
  • All eight assertions and both termination checks are kept. Traced against the head code, they still fail if the iteration-1 response stops being suppressed, a finished request gets an early response, or termination repeats.
  • The removed line is the only waive citing nvbugs/7125734, and the test runs pre-merge on the CPU stages through unittest/_torch/pyexecutor in l0_cpu.yml:118 (file-level cpu_only mark).
  • Not covered: The test was not run here; the before/after results in the description were not reproduced.
  • Not covered: No pre-merge test pipeline has run on this commit yet.

Automated review by trtllm-agent-review. If a point is wrong, reply in its thread. After you push fixes or reply, the bot re-checks the Blockers within a few hours.

@trtllm-agent

Copy link
Copy Markdown
Collaborator Author

/bot run

Signed-off-by: trtllm-agent <296075020+trtllm-agent@users.noreply.github.com>

# Conflicts:
#	tests/integration/test_lists/waives.txt
@trtllm-agent

Copy link
Copy Markdown
Collaborator Author

/bot run

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #86472 [ run ] triggered by Bot. Commit: 08c9269 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #86472 [ run ] completed with state SUCCESS. Commit: 08c9269
/LLM/main/L0_MergeRequest_PR pipeline #71666 completed with status: 'SUCCESS'

CI Report

Link to invocation

@trtllm-agent
trtllm-agent merged commit ea1090d into NVIDIA:main Nov 21, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants