Skip to content

[https://nvbugs/7088470][fix] Restore FP8 prefix-cache accuracy coverage - #22495

Merged
Wanli-Jiang merged 1 commit into
NVIDIA:mainfrom
tensorrt-cicd:repair-bot-v2/nvbug-7088470-r463
Nov 18, 2026
Merged

Wanli-Jiang merged 1 commit into
NVIDIA:mainfrom
tensorrt-cicd:repair-bot-v2/nvbug-7088470-r463

Conversation

@trtllm-agent

@trtllm-agent trtllm-agent commented Nov 18, 2026 •

Copy link
Copy Markdown
Collaborator

Description

Coverage follow-up to #21621, merged as 5007a5be99e8.
The product FP8 routing fix is already on main; this change restores coverage
without duplicating that code.

Remove only the https://nvbugs/7088470 waiver for
accuracy/test_llm_api_pytorch.py::TestQwen3_8_Flash_Next::test_fp8_adp4_mtp3_prefix_cache.
The existing case is registered in
tests/integration/test_lists/test-db/l0_gb300_multi_gpus.yml.
All other waiver entries remain unchanged.

The sibling test_fp8_adp4_mtp3_trtllm_ple_offload waiver
(https://nvbugs/7078608) is intentionally unchanged. The verification below
does not include that case or establish that its reported pre-merge worker exit
was caused by the routing assertion. This PR does not claim to fix that separate
failure.

Fixes https://nvbugs/7088470

Test Coverage

Verified with the waiver removal applied to main snapshot a8b2da508369:

  • The 17 executed predecessor cases from the affected CI shard, followed by accuracy/test_llm_api_pytorch.py::TestQwen3_8_Flash_Next::test_fp8_adp4_mtp3_prefix_cache, run together in one pytest process: 18 passed, no skips, on four GB300 GPUs, using the source waiver file with only this entry removed.
  • CPU-marked routing/layout tests in tests/unittest/_torch/modeling_v2/models/qwen4_exp/qwen3_8_flash_next_nvfp4__sm_103__default/test_routing.py and tests/unittest/_torch/modeling_v2/models/qwen4_exp/qwen3_8_flash_next_fp8__sm_103__unified/test_routing_and_layout.py: 276 passed, no skips; 35 non-CPU cases deselected.
  • The existing accuracy test covers the restored entry. No test assertions, references, or product code change; no new code paths are introduced.

PR Checklist

  • Reviewed the applicable checklist items for this waiver-only change.

Dev Engineer Review

  • The diff removes one line from tests/integration/test_lists/waives.txt. The removed entry is the SKIP waiver for accuracy/test_llm_api_pytorch.py::TestQwen3_8_Flash_Next::test_fp8_adp4_mtp3_prefix_cache (NVBug 7088470).
  • No source code changes. No test code changes. Other waiver entries are unchanged.
  • Risk: the test runs again in CI on GB300 after this change. The PR reports 18 passed, no skips, on four GB300 GPUs, with the waiver removed. These results come from the author. This review did not run the tests, and no CI results for this head were available.
  • The product fix that the test depends on is stated as merged in #21621. This PR does not change that code.

QA Engineer Review

Test files and test-list files changed:

  • Modified: tests/integration/test_lists/waives.txt (1 line removed).

Waivers removed:

  • accuracy/test_llm_api_pytorch.py::TestQwen3_8_Flash_Next::test_fp8_adp4_mtp3_prefix_cache SKIP (https://nvbugs/7088470)

Behavior and coverage:

  • The restored case covers FP8 routing with ADP4, MTP3, and prefix cache on Qwen3-8 Flash Next.
  • The PR reports these results, with the waiver removed:
    • 18 passed, no skips, in one pytest process on four GB300 GPUs. This covers 17 predecessor cases and the restored case.
    • 276 passed, no skips, for the CPU-marked routing and layout tests under tests/unittest/_torch/modeling_v2/models/qwen4_exp/. 35 non-CPU cases were deselected.
  • The test remains registered in tests/integration/test_lists/test-db/l0_gb300_multi_gpus.yml.

Coverage verdict: needs follow-up. The author-reported results pass, but no CI run on the current head was available to confirm them.

Per-File QA Perspective

  • tests/integration/test_lists/waives.txt: The restored case now runs in the GB300 multi-GPU job. QA should confirm that the case passes in CI before the waiver is considered closed.

Remove the waiver for
accuracy/test_llm_api_pytorch.py::TestQwen3_8_Flash_Next::test_fp8_adp4_mtp3_prefix_cache
after the block-FP8 route landed in NVIDIA#21621 (5007a5b).

The case remains registered in
tests/integration/test_lists/test-db/l0_gb300_multi_gpus.yml.
Keep the other entries in tests/integration/test_lists/waives.txt unchanged.

Signed-off-by: trtllm-agent <296075020+trtllm-agent@users.noreply.github.com>
@coderabbitai

coderabbitai Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

The waiver list no longer skips TestQwen3_8_Flash_Next::test_fp8_adp4_mtp3_prefix_cache.

Changes

Qwen3 test waiver

Layer / File(s) Summary
Remove test waiver
tests/integration/test_lists/waives.txt
The skip-list entry for TestQwen3_8_Flash_Next::test_fp8_adp4_mtp3_prefix_cache was removed.

Estimated code review effort: 1 (Trivial) | ~4 minutes

Change: Other

Suggested reviewers: crazydemo, xinhe-nv, larryxfly

Merge Risk: Minimal · up to 8d2bc

This change re-enables one previously waived Qwen3 accuracy test after the underlying routing fix was merged. It touches no product code, and no actionable merge-blocking risk remains.

Pre-merge checks: Passed 4

Check name Status Explanation
Title check Passed The title uses the NVBugs ID and [fix] type in the template format. It names the restored FP8 prefix-cache accuracy coverage, which matches the sole change: removal of the waiver for test_fp8_adp4_mtp…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Description check Passed The description states the purpose: removing only the https://nvbugs/7088470 waiver for test_fp8_adp4_mtp3_prefix_cache, which restores coverage after the FP8 routing fix in #21621. It explains that t…
Full details: Title check

Explanation

The title uses the NVBugs ID and [fix] type in the template format. It names the restored FP8 prefix-cache accuracy coverage, which matches the sole change: removal of the waiver for test_fp8_adp4_mtp3_prefix_cache in waives.txt. It is a bit broader than the one-line waiver removal, but it reflects the main change.

Full details: Description check

Explanation

The description states the purpose: removing only the https://nvbugs/7088470 waiver for test_fp8_adp4_mtp3_prefix_cache, which restores coverage after the FP8 routing fix in #21621. It explains that the sibling waiver (nvbugs/7078608) is intentionally unchanged, and it gives concrete test results (18 passed on four GB300 GPUs; 276 CPU routing/layout tests passed). It also includes a Fixes link and a checklist item. The template's Description and Test Coverage sections are present and substantive. The PR title and the template's PR Checklist items are not addressed beyond one checkbox, so the author should confirm the title format follows the NVBugs pattern and that the checklist items apply.

Finishing Touches

Generate unit tests (beta)

  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Recent review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between eafebd8 and 8d2bcf7.

Files selected for processing: 1

  • tests/integration/test_lists/waives.txt
Files with no reviewable changes: 1

  • tests/integration/test_lists/waives.txt

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 9 remain after this review.

@trtllm-agent-review trtllm-agent-review left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked the un-waive against nvbugs/7088470 and the fix it depends on, #21621 (5007a5b), which is on the base. The failure came from FP8_BLOCK_SCALES checkpoints being auto-routed to the NVFP4 modeling_v2 target on SM103. #21621 now sends them to the FP8 target, and that target accepts this test's dep4 / MTP3 / block-reuse configuration. Posted: 0 blockers, 0 possible blockers, 1 nice to have (inline, on the sibling 7078608 waive).

Comment thread tests/integration/test_lists/waives.txt Resolved
@github-actions

Copy link
Copy Markdown

Automatically added "ci: full pre-merge approved" because this PR has satisfied the required GitHub review approvals. Unresolved review conversations and other required checks remain independent merge requirements.

@Wanli-Jiang

Copy link
Copy Markdown
Collaborator

/bot run --disable-fail-fast

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #85621 [ run ] triggered by Bot. Commit: 8d2bcf7 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #85621 [ run ] completed with state SUCCESS. Commit: 8d2bcf7
/LLM/main/L0_MergeRequest_PR pipeline #70881 completed with status: 'SUCCESS'

CI Report

Link to invocation

@Wanli-Jiang
Wanli-Jiang enabled auto-merge (squash) November 18, 2026 03:14
@Wanli-Jiang
Wanli-Jiang merged commit 9712449 into NVIDIA:main Nov 18, 2026
28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants