-
Notifications
You must be signed in to change notification settings - Fork 2.5k
- #15044 · laikhtewari opened
on Jun 6, 2026 1 - #3148 · juney-nvidia opened
on Mar 29, 2025 5 - #3124 · juney-nvidia opened
on Mar 27, 2025 11
Issues
is:issue state:open
is:issue state:open
Issue creation is restricted in this repository
Search results
[Feature]: Paged / KV-manager-resident storage for the DFlash draft context, replacing the dense buffer
feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality supportSpeculative Decoding<NV>MTP/Eagle/Medusa/Lookahead/Prompt-Lookup-Decoding/Draft-Target-Model/ReDrafter<NV>MTP/Eagle/Medusa/Lookahead/Prompt-Lookup-Decoding/Draft-Target-Model/ReDrafterStatus: Open.#16005 In NVIDIA/TensorRT-LLM;[Performance]: Compact pseudo-KV/topological sparse attention benchmark evidence and backend gap
General perf<NV>Broad performance issues not specific to a particular component<NV>Broad performance issues not specific to a particular componentPytorch<NV>Pytorch backend related issues<NV>Pytorch backend related issuesStatus: Open.#15989 In NVIDIA/TensorRT-LLM;[Bug]: DGX Spark playbook supported models fail in trtllm-serve: Nemotron-3 Nano Omni and GPT-OSS rejected as unsupported due to MODEL_MAP registry mismatch
bugSomething isn't workingSomething isn't workingModel customization<NV>Adding support for new model architectures or variants<NV>Adding support for new model architectures or variantsStatus: Open.#15941 In NVIDIA/TensorRT-LLM;[Feature]: Support multiple input types
feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality supportStatus: Open.#15885 In NVIDIA/TensorRT-LLM;[Bug]: B12x NVFP4 MoE JIT fails on SM120/SM121 with CUDA 12 CuTe DSL payload
Customized kernels<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.Pytorch<NV>Pytorch backend related issues<NV>Pytorch backend related issuesStatus: Open.#15853 In NVIDIA/TensorRT-LLM;[Bug]: Redundant KV host cache tier on DGX Spark unified-memory systems
KV-Cache Managementkv-cache management for efficient LLM inferencekv-cache management for efficient LLM inferenceStatus: Open.#15793 In NVIDIA/TensorRT-LLM;SM120 skip-softmax FMHA support probe misses bridge path
Customized kernels<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.Status: Open.#15791 In NVIDIA/TensorRT-LLM;[Feature]: General cross-instance KV-cache transfer/staging (kv_transfer)
Disaggregated serving<NV>Deploying with separated, distributed components (params, kv-cache, compute). Arch & perf.<NV>Deploying with separated, distributed components (params, kv-cache, compute). Arch & perf.Status: Open.#15735 In NVIDIA/TensorRT-LLM;[Bug]: Clamp very small non-zero temperature values to avoid numerical instability
bugSomething isn't workingSomething isn't workingDecoding/Sampling<NV>Token sampling algorithms in TRTLLM for text gen (top-k, top-p, beam).<NV>Token sampling algorithms in TRTLLM for text gen (top-k, top-p, beam).Status: Open.#15715 In NVIDIA/TensorRT-LLM;[Bug]: Bad Words Missing Space-Prefixed Token Variants for BPE Tokenizers (e.g., GPT-2, Qwen)
bugSomething isn't workingSomething isn't workingStatus: Open.#15706 In NVIDIA/TensorRT-LLM;[DeepSeek-V4] Overlap scheduler + chunked prefill deadlocks in sparse-MLA ctx metadata (device→host sync at _compute_ctx_compressed_position_ids)
Pytorch<NV>Pytorch backend related issues<NV>Pytorch backend related issuesStatus: Open.#15684 In NVIDIA/TensorRT-LLM;[Bug]: FP8 linear cuda_scaled_mm fast path silently disabled on SM121 (DGX Spark GB10)
Customized kernels<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.Status: Open.#15673 In NVIDIA/TensorRT-LLM;