Repository navigation
Conversation
tdophung
force-pushed
the
jax-moe-compound-ep
branch
from
November 19, 2026 04:54
bb2a836 to
9d118d1
Compare
Collaborator
Author
|
/te-ci jax |
1 similar comment
Collaborator
Author
|
/te-ci jax |
tdophung
force-pushed
the
jax-moe-compound-ep
branch
2 times, most recently
from
November 19, 2026 23:39
7eafb63 to
d86af65
Compare
Collaborator
Author
|
/te-ci L1 jax |
1 similar comment
Collaborator
Author
|
/te-ci L1 jax |
Signed-off-by: Jeremy Berchtold <jberchtold@nvidia.com>
Signed-off-by: Teddy Do <tdophung@nvidia.com>
Signed-off-by: Teddy Do <tdophung@nvidia.com>
Signed-off-by: Teddy Do <tdophung@nvidia.com>
Signed-off-by: Teddy Do <tdophung@nvidia.com>
Signed-off-by: Teddy Do <tdophung@nvidia.com>
Signed-off-by: Teddy Do <tdophung@nvidia.com>
Signed-off-by: Teddy Do <tdophung@nvidia.com>
Signed-off-by: Teddy Do <tdophung@nvidia.com>
Signed-off-by: Teddy Do <tdophung@nvidia.com>
Signed-off-by: Teddy Do <tdophung@nvidia.com>
tdophung
force-pushed
the
jax-moe-compound-ep
branch
from
November 20, 2026 02:23
d86af65 to
7501b01
Compare
Collaborator
Author
|
/te-ci L1 jax |
tdophung
marked this pull request as ready for review
November 20, 2026 02:28
Contributor
[Medium impact] The PR appears safe to merge; no new actionable issues were found. SummaryAdds compound EP axes through
Diagram%%{init: {'theme': 'neutral'}}%%
flowchart LR
R[MeshResource] --> B[EP bootstrap]
R --> M[MoE input layout]
R --> O[Default EP output layout]
B --> G[Compound EP groups]
M --> D[Dispatch and expert work]
G --> D
D --> C[Combine and backward]
O --> C
Reviews (3) · Last reviewed commit: "Check the standalone EP round trip witho..." · Reviewed by Greptile |
Signed-off-by: Teddy Do <tdophung@nvidia.com>
Collaborator
Author
|
/te-ci L1 jax |
Signed-off-by: Teddy Do <tdophung@nvidia.com>
Collaborator
Author
|
/te-ci L1 jax |
Collaborator
Author
|
/te-ci jax |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
The TE MoEBlock must know when the expert-parallel (EP) group spans several mesh axes, for example EP over
("expert", "tensor")when tensor parallelism is folded into EP for the MoE region (EP+TP+FSDP layouts). This PR letsMeshResource.ep_resourcename a tuple of mesh axes and makes the MoE path and the EP bootstrap resolve EP, DP and FSDP the same way from oneMeshResource.It also fixes the EP-group count on meshes with both DP and FSDP: the bootstrap now counts groups over every non-EP outer axis, which a dp x fsdp x ep mesh needs.
This PR supersedes #3462. It carries that PR's net changes by Jeremy Berchtold, rebased on current main (which now includes #3883), followed by follow-up fixes. Four of #3462's commits that cancel each other out (a change and its revert) are not replayed.
Type of change
Changes
MeshResource.ep_resourcemay be a tuple of mesh axes (compound EP); the MoEBlock andmoe()handle compound EP, including with FSDP and with rank-2 inputs.MeshResource:ep_bootstraptakesmesh_resource=(explicit, else the global resource), and the standalone EP ops resolve EP from it._fsdp_outer_axesfrom [JAX] Control MeshResource at API call granularity for Dense APIs #3883; the group count covers every non-EP outer axis.ep_bootstraprejects amesh_resourcethat is not aMeshResource.Checklist:
Co-authored-by: Jeremy Berchtold jberchtold@nvidia.com