Skip to content

[JAX] Run TE EP Moe tests on Hopper - #3908

Open
tdophung wants to merge 5 commits into
NVIDIA:mainfrom
tdophung:te-jax-ep-moe-hopper
Open

tdophung wants to merge 5 commits into
NVIDIA:mainfrom
tdophung:te-jax-ep-moe-hopper

Conversation

@tdophung

Copy link
Copy Markdown
Collaborator

Description

tests/jax/test_te_ep_moe.py skips the whole file below sm_100, because its gate says grouped GEMM needs Blackwell. That is no longer the case: the v2 grouped GEMM supports BF16 on SM90+, and only MXFP8 needs SM100+. This PR lowers the gate to sm_90 and skips the MXFP8 cases wherever is_scaling_mode_supported reports that MXFP8 is unsupported, so Hopper runs the BF16 part of the suite.

On H100 the only new failure was test_forward[*-bf16-sigmoid-bias-strong], and it is a bug in the test's reference, not in TE. On sm_90, XLA splits the reference's gate GEMM along K with fp32 output and, because excess precision is allowed, skips the bf16 rounding that TE applies, so when two experts are within one bf16 ulp of each other, the reference and TE can route a token differently. Rounding the reference logits with jax.lax.reduce_precision fixes that, as _kernel_boundary in tests/jax/test_mhc.py does, and it changes nothing on Blackwell, where XLA already keeps the bf16 output.

Type of change

  • Documentation change (change only to the documentation, either a fix or a new content)
  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Infra/Build change
  • Code refactoring

Changes

All of the changes are in tests/jax/test_te_ep_moe.py:

  • The file gate is now sm_90+ instead of sm_100+, and its comment says what each dtype needs.
  • The MXFP8 cases and MXFP8-only tests skip based on is_scaling_mode_supported(ScalingMode.MXFP8_1D_SCALING) instead of a compute capability check.
  • The reference rounds its gate logits to the input dtype.

Checklist:

  • I have read and followed the contributing guidelines
  • The functionality is complete
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

Signed-off-by: Teddy Do <tdophung@nvidia.com>
Signed-off-by: Teddy Do <tdophung@nvidia.com>
Signed-off-by: Teddy Do <tdophung@nvidia.com>
@tdophung

Copy link
Copy Markdown
Collaborator Author

/te-ci L1 jax

@greptile-apps

greptile-apps Bot commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Low impact] No new blocking issue was found in these test changes.

Summary

Enables BF16 MoE tests on Hopper while keeping MXFP8 tests behind support checks.

  • Hopper now runs BF16 expert-parallel MoE tests when its software supports them.
  • The MoE reference rounds gate logits before it chooses experts.

Reviews (3) · Last reviewed commit: "Merge branch 'main' into te-jax-ep-moe-h..." · Reviewed by Greptile

Comment thread tests/jax/test_te_ep_moe.py Resolved
Signed-off-by: Teddy Do <tdophung@nvidia.com>
@tdophung

Copy link
Copy Markdown
Collaborator Author

/te-ci L1 jax

@tdophung tdophung changed the title [JAX] Run the TE EP MoE tests on Hopper [JAX] Run TE EP Moe tests on Hopper Nov 21, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant