Repository navigation
Conversation
Signed-off-by: Teddy Do <tdophung@nvidia.com>
Signed-off-by: Teddy Do <tdophung@nvidia.com>
Signed-off-by: Teddy Do <tdophung@nvidia.com>
Collaborator
Author
|
/te-ci L1 jax |
Contributor
[Low impact] No new blocking issue was found in these test changes. SummaryEnables BF16 MoE tests on Hopper while keeping MXFP8 tests behind support checks.
Reviews (3) · Last reviewed commit: "Merge branch 'main' into te-jax-ep-moe-h..." · Reviewed by Greptile |
Signed-off-by: Teddy Do <tdophung@nvidia.com>
Collaborator
Author
|
/te-ci L1 jax |
8 of 13 tasks
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
tests/jax/test_te_ep_moe.pyskips the whole file below sm_100, because its gate says grouped GEMM needs Blackwell. That is no longer the case: the v2 grouped GEMM supports BF16 on SM90+, and only MXFP8 needs SM100+. This PR lowers the gate to sm_90 and skips the MXFP8 cases whereveris_scaling_mode_supportedreports that MXFP8 is unsupported, so Hopper runs the BF16 part of the suite.On H100 the only new failure was
test_forward[*-bf16-sigmoid-bias-strong], and it is a bug in the test's reference, not in TE. On sm_90, XLA splits the reference's gate GEMM along K with fp32 output and, because excess precision is allowed, skips the bf16 rounding that TE applies, so when two experts are within one bf16 ulp of each other, the reference and TE can route a token differently. Rounding the reference logits withjax.lax.reduce_precisionfixes that, as_kernel_boundaryintests/jax/test_mhc.pydoes, and it changes nothing on Blackwell, where XLA already keeps the bf16 output.Type of change
Changes
All of the changes are in
tests/jax/test_te_ep_moe.py:is_scaling_mode_supported(ScalingMode.MXFP8_1D_SCALING)instead of a compute capability check.Checklist: