Skip to content

[TRTLLM-17362][feat] Integrate Beam with standard serving and public evaluation - #21473

Open
Mgluhovskoi wants to merge 53 commits into
NVIDIA:mainfrom
Mgluhovskoi:mergeback/beam-3-trtllm-17362
Open

Mgluhovskoi wants to merge 53 commits into
NVIDIA:mainfrom
Mgluhovskoi:mergeback/beam-3-trtllm-17362

Conversation

@Mgluhovskoi

@Mgluhovskoi Mgluhovskoi commented Nov 2, 2026 •

Copy link
Copy Markdown
Collaborator

Description

Integrate the Beam 52-layer NVFP4 baseline with standard LLM() and trtllm-serve paths: ModelingV2 routing and engine adapter, registered beam_nvfp4 checkpoint loader, tokenizer, Harmony parser, serving documentation and the public GSM8K evaluator.

Part 3/4 of TRTLLM-17024, implementing TRTLLM-17362. Depends on #21471 and #21472.

The runnable target is introduced here alongside its engine-admission tests and named TestModelingV2BeamFullModelNvfp4Sm103Tp4 accuracy gate. Earlier parts contain directly constructed components only. Component corrections are inherited, and the loader guard accepts the documented checkpoint_format="beam_nvfp4" path while rejecting incompatible explicit loaders.

The baseline is TP4 on four GB300s, NVFP4 experts, BF16 KV, eager execution and overlap disabled. Speculative decoding, FP8 KV, disaggregated serving and other optimizations are outside this baseline.

The evaluator stays with the named Beam serving gate so the recipe and its acceptance test can be reviewed together. It uses lm-eval’s local-chat-completions for request construction and scheduling. Beam also needs the raw finish reason and token usage to detect truncation, the final answer channel for scoring, and per-question records to resume interrupted runs. Compare scores using the same pinned Beam protocol; the repository’s standard GSM8K reference uses different evaluation settings.

Review order

#21471 is merged. #21472 → #21473 → #21474 remain the review order; all target main. This PR includes #21472’s tests plus the serving integration.

Incremental part-3 diff

Validation

At 62a184ba5383e9449f6ee906b13517bf045b6215:

  • 120 host-side component, engine-contract and static ModelingV2 checks pass, including the named-gate requirement. Native dependencies are stubbed in this host check.
  • 79 public-evaluator/gate-wiring CPU tests pass.
  • Actual parser module passes 9 one-shot cases and 646 streaming splits; 97 attention-call keyword bindings match the shared interface.
  • Pre-commit and AST test-list validation pass.

The named gate starts trtllm-serve with modeling_v2: require and runs the pinned public first-100 GSM8K protocol: at least 95 correct, with execution/parsing/truncation failures rejected. It is a manual four-GB300 qualification with a 45-minute cap, not an L0 or full-dataset CI run. Authorized checkpoint and dataset inputs must be staged externally; missing inputs fail.

The gate has not been executed on a matching GPU build for this corrected extraction. Native accuracy/timing qualification is required before scheduling it in LR/nightly or claiming this baseline qualified. Remaining human review, matching-build GPU validation and required CI/code-owner approval remain outstanding. No checkpoint weights or private evidence are included.

November 12 rebase

Rebased onto merged #21471 (08b8c22500), consolidating the remaining scoped changes after its squash merge. The validated four-file test/CI-comment patch is included. Beam implementation and test Python files are unchanged by the rebase; current upstream registrations and parser definitions are preserved. Each rebased branch passed 81 static CI-selection/CBTS checks and pre-commit. This serving branch also passed 38 host-adapted engine checks and six host-adapted checkpoint-loader checks. These host checks do not replace matching-build native GPU CI. Prior native measurements above/below describe their recorded snapshots, not a green result for this rebased head.

Dev Engineer Review

Adds Beam support across the PyTorch backend and the ModelingV2 serving path. The changes include Beam model components and operators, TP4 NVFP4 checkpoint loading and routing, tokenizer and Harmony parser support, serving documentation, and a pinned GSM8K evaluator.

The engine route accepts a narrow four-rank SM 10.3 NVFP4 configuration and refuses unsupported serving options. The documented baseline uses BF16 KV cache and disables overlap scheduling. Speculative decoding, FP8 KV cache, and disaggregated serving are outside its scope.

The evaluator supports managed-server and existing-endpoint modes, resumable records, model identity checks, and explicit failure verdicts. The manual gate requires at least 95 correct answers on the pinned first 100 GSM8K questions. The supplied context says matching-build GPU qualification remains outstanding. It also reports one successful PR job and one failed merge-request pipeline; investigation and a rerun remain outstanding.

QA Engineer Review

The change adds tests for Beam operators and components, checkpoint layouts and loading, routing and engine contracts, diagnostic behavior, Harmony parsing, evaluator protocol and records, and the accuracy gate. The l0_cpu.yml list includes component-contract, engine-contract, baseline-gate, diagnostic-mode, loading-contract, and serving-parser tests. It does not include tests/unittest/tools/test_nvfp4_serving_gsm8k_first100.py. The manual GSM8K gate is listed in manual_gb300_beam_baseline_gsm8k_first100.yml for the four-GB300 profile, with a 45-second test-list timeout.

Coverage verdict: needs follow-up. CPU contract tests cover key evaluator and serving paths, but they do not establish GPU compatibility. Run the manual accuracy gate on a matching GPU build and investigate the reported failed merge-request pipeline.

Per-File QA Perspective

Documentation and evaluator

  • docs/source/deployment-guide/deployment-guide-for-beam-on-trtllm.md: Verify the documented serving configuration and manual accuracy criteria against runtime behavior.
  • docs/source/deployment-guide/index.rst: Adds the Beam guide to the deployment documentation tree.
  • examples/llm-api/eval_gsm8k.py: Verify CLI validation, endpoint and managed-server modes, runner arguments, and exit codes.
  • examples/llm-api/gsm8k_eval/README.md: Check that usage and failure documentation matches evaluator behavior.
  • examples/llm-api/gsm8k_eval/__init__.py: Adds package documentation; no runtime behavior change is described.
  • examples/llm-api/gsm8k_eval/manifest.json: Verify the pinned dataset, question and exemplar selection, and recorded hashes.
  • examples/llm-api/gsm8k_eval/protocol.py: Verify manifest validation, prompt construction, selection, and scoring.
  • examples/llm-api/gsm8k_eval/records.py: Verify reply classification, ledger validation and resume behavior, and verdict rules.
  • examples/llm-api/gsm8k_eval/runner.py: Verify endpoint and managed-server execution, model consistency, resume identity, and interruption handling.
  • examples/llm-api/gsm8k_eval/server.py: Verify server startup, address reporting, process cleanup, and optional GPU memory polling.

ModelingV2 operators and Beam model

  • tensorrt_llm/_torch/_experimental/modeling_v2/__init__.py: Verify the resolver documentation reflects its import behavior.
  • tensorrt_llm/_torch/_experimental/modeling_v2/catalog/activation/beam_logit_softcap.py, flashinfer_silu_and_mul.py, and swiglu_fp32.py: Verify operator formulas, dtype and shape constraints, and new SM 103 cases.
  • tensorrt_llm/_torch/_experimental/modeling_v2/catalog/attention/beam_neox_rope.py, beam_nope_query_scale.py, beam_output_gate.py, and thop_attention.py: Verify RoPE and query scaling, output gating, attention windows, and FMHA case configurations.
  • tensorrt_llm/_torch/_experimental/modeling_v2/catalog/gemm/cublas_mm.py, catalog/moe/beam_dispatch_combine.py, beam_grouped_topk_router.py, catalog/norm/beam_rmsnorm_fp32.py, flashinfer_rmsnorm.py, and catalog/residual/beam_depth_residual.py: Verify projection coverage, routing and accumulation order, normalization, and residual scaling.
  • tensorrt_llm/_torch/_experimental/modeling_v2/catalog/residual/__init__.py: Adds package metadata; no runtime behavior change is described.
  • tensorrt_llm/_torch/_experimental/modeling_v2/models/beam/README.md: Check documented target constraints and unsupported serving features against implementation.
  • tensorrt_llm/_torch/_experimental/modeling_v2/models/beam/__init__.py and the Beam component package __init__.py files: These add package metadata; no runtime behavior change is described.
  • tensorrt_llm/_torch/_experimental/modeling_v2/models/beam/dense_block_bf16__sm_103__tp1/block.py, moe_bf16__sm_103__tp1/moe.py, moe_expert_bf16__sm_103__tp1/expert.py, moe_router_bf16__sm_103__tp1/router.py, and sparse_block_bf16__sm_103__tp1/block.py: Verify supported configurations, weight-loading guards, attention and routing behavior, and CUDA device checks.
  • tensorrt_llm/_torch/_experimental/modeling_v2/models/beam/small_model_prefill_bf16__sm_103__tp1/model.py and generation.py: Verify cache lifecycle, prefix reuse, weight-change detection, and greedy generation.
  • tensorrt_llm/_torch/_experimental/modeling_v2/models/beam/full_model_nvfp4__sm_103__tp4/_attention.py, component.py, component_bf16.py, dispatch_combine.py, expert.py, layout.py, layout_bf16.py, loader.py, and loader_bf16.py: Verify attention metadata and cache checks, rank-local checkpoint reads, quantization layout, collective behavior, and load-failure handling.
  • tensorrt_llm/_torch/_experimental/modeling_v2/models/beam/full_model_nvfp4__sm_103__tp4/model.py and modeling.py: Verify full-model execution, prefill/decode dispatch, request restrictions, and logit selection.
  • tensorrt_llm/_torch/_experimental/modeling_v2/models/beam/full_model_nvfp4__sm_103__tp4/weights.py and models/beam/routing.py: Verify checkpoint-loader integration and engine admission/refusal behavior.
  • tensorrt_llm/_torch/_experimental/modeling_v2/models/__init__.py: Verify importing the package loads Beam routing without unintended registration or import effects.
  • tensorrt_llm/_torch/models/checkpoints/__init__.py, tensorrt_llm/_torch/models/checkpoints/beam/checkpoint_loader.py, and tokenizer.py: Verify public exports, beam_nvfp4 registration, and tokenizer loading.
  • tensorrt_llm/_torch/models/checkpoints/beam/__init__.py: Adds package metadata; no runtime behavior change is described.
  • tensorrt_llm/_torch/models/modeling_beam.py: Verify model computation, configuration validation, and strict checkpoint mapping and loading.
  • tensorrt_llm/_torch/pyexecutor/config_utils.py: Verify Beam config construction preserves the checkpoint path and supplied settings.
  • tensorrt_llm/llmapi/reasoning_parser.py: Verify complete and streamed Harmony channel parsing, including termination markers.

Tests and test lists

  • tests/integration/defs/accuracy/modeling_v2/test_beam.py: Adds the manual first-100 GSM8K gate. It is listed in the manual GB300 test list.
  • tests/integration/test_lists/test-db/manual_gb300_beam_baseline_gsm8k_first100.yml: Adds the manual gate selector for the four-GB300 profile. Verify its selectors and timeout against the test.
  • tests/integration/test_lists/test-db/l0_cpu.yml: Adds CPU component-contract, engine-contract, baseline-gate, diagnostic, loading, and serving tests. The evaluator protocol/records test is not listed.
  • tests/unittest/tools/test_nvfp4_serving_gsm8k_first100.py: Covers protocol, scoring, dataset checks, reply handling, and ledger behavior with synthetic data. It is not listed in l0_cpu.yml.
  • tests/unittest/tools/test_beam_baseline_accuracy_gate.py: Covers gate arguments, failure propagation, and checkpoint-path validation. It is listed in l0_cpu.yml.
  • tests/unittest/_torch/modeling_v2/models/beam/test_component_contracts.py and test_engine_contracts.py: Cover cache and checkpoint contracts, request validation, engine-option refusal, and checkpoint-loader routing. Both are listed in l0_cpu.yml.
  • tests/unittest/_torch/modeling_v2/models/beam/test_loading_contracts.py and test_diagnostic_mode.py: Cover loading failures, weight reload behavior, and rank agreement. Both are listed in l0_cpu.yml.
  • tests/unittest/_torch/modeling_v2/models/beam/full_model_nvfp4__sm_103__tp4/test_serving.py: Covers parser registration and pretrained config loading. It is listed in l0_cpu.yml.
  • tests/unittest/_torch/modeling_v2/models/beam/full_model_nvfp4__sm_103__tp4/test_batching.py: Covers batched attention metadata and logit-row selection. No test-list entry is shown.
  • tests/unittest/_torch/modeling_v2/models/beam/checkpoint_layout_bf16__sm_103__tp4/test_checkpoint_layout.py and checkpoint_layout_nvfp4__sm_103__tp4/test_checkpoint_layout_nvfp4.py: Cover layout partitioning, checkpoint metadata, rank slices, and CUDA loading paths. No test-list entries are shown.
  • tests/unittest/_torch/modeling_v2/models/beam/_beam_component_support.py: Adds shared test fixtures and evidence helpers; it is not a test module listed in a test list.
  • tests/unittest/_torch/modeling_v2/models/beam/dense_block_bf16__sm_103__tp1/_dense_block_reference.py: Adds a Torch reference for dense-block calculations; no test-list entry is shown.
  • tests/unittest/_torch/modeling_v2/models/beam/dense_block_bf16__sm_103__tp1/test_dense_block.py: Covers dense-block reference comparisons, restrictions, execution paths, and KV-cache writes. No test-list entry is shown.
  • tests/unittest/_torch/modeling_v2/models/beam/moe_bf16__sm_103__tp1/_moe_reference.py: Adds a Torch reference for MoE routing and expert computations; no test-list entry is shown.
  • tests/unittest/_torch/modeling_v2/models/beam/moe_bf16__sm_103__tp1/test_moe.py: Covers MoE routing, numerical behavior, configuration, weights, and execution paths. No test-list entry is shown.
  • tests/unittest/_torch/modeling_v2/models/beam/moe_dispatch_combine_bf16__sm_103__tp1/_dispatch_combine_cases.py: Adds generated fixtures and a golden dispatch/combine implementation; no test-list entry is shown.
  • tests/unittest/_torch/modeling_v2/models/beam/moe_dispatch_combine_bf16__sm_103__tp1/moe/test_modeling_v2_beam_dispatch_combine.py and test_dispatch_combine.py: Cover operator certification and component routing, output ordering, and input immutability. No test-list entries are shown.
  • tests/unittest/_torch/modeling_v2/models/beam/moe_expert_bf16__sm_103__tp1/_expert_cases.py: Adds numerical controls, fixtures, and a golden expert implementation; no test-list entry is shown.
  • tests/unittest/_torch/modeling_v2/models/beam/moe_expert_bf16__sm_103__tp1/activation/test_modeling_v2_swiglu_fp32.py: Covers SwiGLU reference and precision behavior, with validation guards. No test-list entry is shown.
  • tests/unittest/_torch/modeling_v2/models/beam/moe_expert_bf16__sm_103__tp1/test_expert.py: Covers expert loading, validation, numerical stages, and execution behavior. No test-list entry is shown.
  • tests/unittest/_torch/modeling_v2/models/beam/moe_router_bf16__sm_103__tp1/_router_cases.py: Adds router fixtures and comparison helpers; no test-list entry is shown.
  • tests/unittest/_torch/modeling_v2/models/beam/moe_router_bf16__sm_103__tp1/moe/test_modeling_v2_beam_grouped_topk_router.py and test_router.py: Cover router reference comparisons, routing fixtures, validation, and numerical controls. No test-list entries are shown.
  • tests/unittest/_torch/modeling_v2/models/beam/small_model_prefill_bf16__sm_103__tp1/test_beam_logit_softcap.py: Covers softcap operator behavior and validation. No test-list entry is shown.
  • tests/unittest/_torch/modeling_v2/models/beam/small_model_prefill_bf16__sm_103__tp1/test_cached_generation.py: Covers generation lifecycle, cache writes, prefix reuse, and weight-change detection. No test-list entry is shown.
  • tests/unittest/_torch/modeling_v2/models/beam/small_model_prefill_bf16__sm_103__tp1/test_small_model_prefill.py: Covers prefill validation, execution order, attention behavior, and request isolation. No test-list entry is shown.
  • tests/unittest/_torch/modeling_v2/models/beam/sparse_block_bf16__sm_103__tp1/_sparse_block_reference.py: Adds a Torch reference for sparse-block calculations; no test-list entry is shown.
  • tests/unittest/_torch/modeling_v2/models/beam/sparse_block_bf16__sm_103__tp1/test_sparse_block.py: Covers sparse-block output, routing, configuration, and cache behavior. No test-list entry is shown.
  • tests/unittest/_torch/modeling_v2/attention/test_modeling_v2_beam_neox_rope.py, test_modeling_v2_beam_nope_query_scale.py, and test_modeling_v2_beam_output_gate.py: Cover operator references, shape and dtype behavior, precision, and validation guards. No test-list entries are shown.
  • tests/unittest/_torch/modeling_v2/attention/test_modeling_v2_thop_attention.py: Extends attention reference and paged-context coverage for Beam sliding and full attention. No test-list entry is shown.
  • tests/unittest/_torch/modeling_v2/residual/test_modeling_v2_beam_depth_residual.py: Covers residual coefficients, reference output, and validation guards. No test-list entry is shown.
  • tests/unittest/_torch/modeling_v2/models/beam/checkpoint_layout_bf16__sm_103__tp4/norm/test_modeling_v2_beam_rmsnorm_fp32.py: Covers RMSNorm reference precision, output properties, and validation. No test-list entry is shown.

@Mgluhovskoi

Copy link
Copy Markdown
Collaborator Author

/bot run

@coderabbitai

coderabbitai Bot commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • Resume reviews
  • Trigger review

Walkthrough

This pull request adds Beam ModelingV2 implementations for SM 103, including BF16 components and a four-rank NVFP4 target. It adds serving registration, a pinned GSM8K evaluator with endpoint and managed-server modes, and a manual first-100 accuracy qualification.

Changes

Beam ModelingV2 and serving

Layer / File(s) Summary
Beam operators and catalog coverage
tensorrt_llm/_torch/_experimental/modeling_v2/catalog/*, tests/unittest/_torch/modeling_v2/attention/*, tests/unittest/_torch/modeling_v2/residual/*
Adds Beam activation, attention, MoE, normalization, and residual operations, with catalog cells and tests.
BF16 components and full-model execution
tensorrt_llm/_torch/_experimental/modeling_v2/models/beam/..., tests/unittest/_torch/modeling_v2/models/beam/...
Adds BF16 dense and sparse blocks, MoE components, a six-layer model with cached generation, TP4 checkpoint layouts and loaders, and a 52-layer NVFP4 model. Tests cover operators, components, loading, cache behavior, and model contracts.
Serving integration and qualification
tensorrt_llm/_torch/models/*, tensorrt_llm/llmapi/reasoning_parser.py, docs/source/deployment-guide/*, tests/integration/...
Adds Beam configuration, checkpoint loading, tokenization, routing, and Harmony parsing. Documents and tests the four-GB300 GSM8K baseline.
Attention catalog coverage
tensorrt_llm/_torch/_experimental/modeling_v2/catalog/attention/thop_attention.py, tensorrt_llm/_torch/_experimental/modeling_v2/catalog/gemm/cublas_mm.py, tensorrt_llm/_torch/_experimental/modeling_v2/catalog/norm/flashinfer_rmsnorm.py, tensorrt_llm/_torch/_experimental/modeling_v2/catalog/activation/flashinfer_silu_and_mul.py, tests/unittest/_torch/modeling_v2/attention/test_modeling_v2_thop_attention.py
Adds Beam workload cells for attention, projections, normalization, and activation. Attention test helpers now pass window and block limits from each cell.

GSM8K evaluator

Layer / File(s) Summary
Pinned protocol and dataset checks
examples/llm-api/gsm8k_eval/protocol.py, examples/llm-api/gsm8k_eval/manifest.json, tests/unittest/tools/test_nvfp4_serving_gsm8k_first100.py
Pins dataset rows, exemplar selection, prompt construction, requests, scoring, and integrity hashes. Tests exercise protocol validation and answer scoring.
Evaluation execution and records
examples/llm-api/eval_gsm8k.py, examples/llm-api/gsm8k_eval/runner.py, examples/llm-api/gsm8k_eval/server.py, examples/llm-api/gsm8k_eval/records.py
Adds endpoint and managed-server execution, resumable JSONL records, run identity checks, verdicts, reporting, process cleanup, and CLI exit statuses.
Evaluator and deployment documentation
examples/llm-api/gsm8k_eval/README.md, docs/source/deployment-guide/*, tests/integration/defs/accuracy/modeling_v2/test_beam.py
Documents evaluator modes and the manual baseline. The gate runs 100 questions with concurrency 1, a 2700-second cap, and a 95-answer minimum.

Estimated code review effort: 5 (Critical) | ~180 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant ModelingV2Target
  participant BeamFullModel
  participant BeamComponent
  participant KVCache
  ModelingV2Target->>BeamFullModel: dispatch prefill or decode
  BeamFullModel->>BeamComponent: execute model layers
  BeamComponent->>KVCache: read and append attention state
  BeamComponent-->>ModelingV2Target: return selected logits
Loading
sequenceDiagram
  participant EvalCLI
  participant EvalRunner
  participant ServingEndpoint
  participant ResultLedger
  EvalCLI->>EvalRunner: validate protocol and run settings
  EvalRunner->>ServingEndpoint: resolve model and submit questions
  ServingEndpoint-->>EvalRunner: return completion replies
  EvalRunner->>ResultLedger: append records and persist verdict
Loading

Suggested reviewers: wanli-jiang, brnguyen2

Merge Risk: Moderate · previous review 1c643

The documentation build is blocked, mixed batches can receive incorrect logits, and the accuracy gate can pass despite a parsing failure. These issues should be addressed before merging.

Pre-merge checks: Passed 4

Check name Status Explanation
Title check Passed The title clearly identifies the feature, ticket, and primary integration work.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Description check Passed The description supplies substantive content: it states the purpose (integrating the Beam 52-layer NVFP4 baseline with LLM() and trtllm-serve), the scope and exclusions, the review order and dependenc…
Full details: Description check

Explanation

The description supplies substantive content: it states the purpose (integrating the Beam 52-layer NVFP4 baseline with LLM() and trtllm-serve), the scope and exclusions, the review order and dependencies, validation results, and outstanding items. It does not follow the template's required headings exactly: the PR title format is not shown in the description, the '## Test Coverage' section is missing as a distinct heading (test results appear under '## Validation'), and the PR Checklist is absent. These gaps are non-critical relative to the substance of the description, so the check passes.

Finishing Touches

Generate unit tests (beta)

  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

@coderabbitai coderabbitai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Fix comments: Fix CodeRabbit comments on this PR

Comment thread examples/llm-api/gsm8k_eval/README.md Resolved
Comment thread examples/llm-api/gsm8k_eval/records.py Resolved
@yuxianq

yuxianq commented Nov 2, 2026 •

Copy link
Copy Markdown
Collaborator

@kris1025, could you help to review this PR as trt-llm-torch-attention-devs? Thanks~

Assignment explanation
Componentsmodules/attention (25%); modules/qk-rope-gating (20%); fmha/flashinfer-trtllm-gen (10%); integration/attention-tests (25%); integration/model-config (5%); unmapped (15%)
Tagsalgorithm:dense (100%); representation:gqa (100%); phase:prefill (60%); phase:decode (40%); features:rope (40%); features:qk-norm (20%); features:paged-kv (40%); context:llm (100%)
PR effort8 points — The supplied snapshot includes model introduction, not only serving integration. Attention review requires tracing distinct numerical contracts, sliding/full GQA, cached batch spans and FMHA eligibility across files; public evaluation and other non-attention volume do not drive the score.
ImportanceImportant — Added model.py/modeling.py introduce the 52-layer Beam target in this supplied diff. New RoPE/query-scale/gate contracts and build-dependent paged-context fallback selection affect attention correctness and execution behavior.
Affinity (0–1)0.382 component/tag match; 0.287 after PIC preference
Outstanding points at selection40 → 40 (+0)
Points already reserved on this PR8
Effective weight1
Completed review points (preceding 168 hours)24
Combined points after assignment64
Demand-derived target60.00 points (soft share)
Additional fair share before assignment0.00 points
Share explanationWhole PR / qualification or coverage constraint exceeds a soft share; assignment is allowed.
Selection ruleaffinity with 0.3-share guard
Assignment scope (kris1025)all

Affinity and workload are snapshots at selection, not live counters. Existing reservations on this PR are counted once.

Signed-off-by: Maxim Gluhovskoi <mgluhovskoi@nvidia.com>
@Mgluhovskoi
Mgluhovskoi force-pushed the mergeback/beam-3-trtllm-17362 branch from 3e12ed7 to 193d092 Compare November 19, 2026 20:54
Signed-off-by: Maxim Gluhovskoi <mgluhovskoi@nvidia.com>
@Mgluhovskoi Mgluhovskoi added the api-compatible Accepted LLM API contract change that is backwards-compatible label Nov 19, 2026
@Mgluhovskoi

Copy link
Copy Markdown
Collaborator Author

/bot run

Signed-off-by: Maxim Gluhovskoi <mgluhovskoi@nvidia.com>
Signed-off-by: Maxim Gluhovskoi <mgluhovskoi@nvidia.com>
Signed-off-by: Maxim Gluhovskoi <mgluhovskoi@nvidia.com>
Signed-off-by: Maxim Gluhovskoi <mgluhovskoi@nvidia.com>
Signed-off-by: Maxim Gluhovskoi <mgluhovskoi@nvidia.com>
Signed-off-by: Maxim Gluhovskoi <mgluhovskoi@nvidia.com>
Signed-off-by: Maxim Gluhovskoi <mgluhovskoi@nvidia.com>
@Mgluhovskoi

Copy link
Copy Markdown
Collaborator Author

/bot run

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #86195 [ run ] triggered by Bot. Commit: cfd5e0b Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #86195 [ run ] completed with state SUCCESS. Commit: cfd5e0b
/LLM/main/L0_MergeRequest_PR pipeline #71401 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

Comment thread jenkins/L0_Test.groovy Resolved
Signed-off-by: Maxim Gluhovskoi <mgluhovskoi@nvidia.com>
Signed-off-by: Maxim Gluhovskoi <mgluhovskoi@nvidia.com>
@Mgluhovskoi
Mgluhovskoi requested a review from chzblych November 20, 2026 18:25
@Mgluhovskoi

Copy link
Copy Markdown
Collaborator Author

/bot run

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #86431 [ run ] triggered by Bot. Commit: 75e0c86 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #86431 [ run ] completed with state SUCCESS. Commit: 75e0c86
/LLM/main/L0_MergeRequest_PR pipeline #71627 completed with status: 'UNSTABLE'

CI Report

⚠️ Multi-GPU Label Required:
Multi-GPU tests require the ci: full pre-merge approved label on this PR. Either:

  • Wait for the PR to be fully approved — the label is added automatically once approval is complete. Having unresolved open comments is fine, or
  • If needed, ask a member of NVIDIA/trt-llm-ci-approvers to add the label manually.
    Then re-trigger CI with the same bot command (no rebase needed).

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

Signed-off-by: Maxim Gluhovskoi <mgluhovskoi@nvidia.com>
@Mgluhovskoi

Copy link
Copy Markdown
Collaborator Author

/bot run

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #86496 [ run ] triggered by Bot. Commit: 241defa Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #86496 [ run ] completed with state SUCCESS. Commit: 241defa
/LLM/main/L0_MergeRequest_PR pipeline #71687 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation


# Tiny local Wan/FLUX checkpoints; one CUDA GPU, no pretrained model download.
unittest/_torch/visual_gen/test_sleep_non_h3.py
accuracy/modeling_v2/beam/full_model_nvfp4__sm_103__tp4/test_accuracy.py::TestModelingV2BeamFullModelNvfp4Sm103Tp4::test_gsm8k_first100 TIMEOUT (45)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpick: could you move the accuracy test to a more proper location in this test list? I think the test is not Van/FLUX related.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Agent Code Review: High Risk api-compatible Accepted LLM API contract change that is backwards-compatible

Projects

None yet

Development

Successfully merging this pull request may close these issues.