Repository navigation
[JAX] Add EP borrowed-comm path via XLA FFI collectives extension - #3452
Conversation
29d4ed6 to
e081e4c
Compare
|
|
/te-ci L1 JAX |
jberchtold-nvidia
left a comment
There was a problem hiding this comment.
Took a review pass and everything LGTM besides existing comments from Teddy. I appreciate the thorough version handling in this between pre- and post- this feature landing in XLA. Thanks!
There was a problem hiding this comment.
Can we use the bultin version of this? https://github.com/openxla/xla/blob/main/xla/ffi/api/collectives_api.h
There was a problem hiding this comment.
Thanks Michael, good suggestion.
XLA has an equivalent (xla::ffi::Collectives/Communicator in collectives_ffi.h), but there is a blocker: jaxlib doesn't package that header yet. Besides, XLA's version has no Support() override, so it requires an exact major.minor match. Left a TODO in the file - I will send the fixes to jaxlib and XLA, then will drop our copy.
Deferring for now since this is not a blocker.
Signed-off-by: Phuong Nguyen <phuonguyen@nvidia.com>
…truct init Signed-off-by: Phuong Nguyen <phuonguyen@nvidia.com>
e081e4c to
29d4ed6
Compare
|
/te-ci L1 JAX |
Signed-off-by: Phuong Nguyen <phuonguyen@nvidia.com>
29d4ed6 to
7f5ebc3
Compare
|
/te-ci L1 JAX |
Description
Adds an EP bootstrap path that borrows JAX/XLA's own NCCL communicator instead of self-hosting one via
ncclCommInitRank, avoiding a second communicator per EP domain. Auto-selected when the XLA-borrowed NCCL comm is available (build + installed JAX/jaxlib support it); otherwise falls back to self-hosted NCCL.Type of change
Changes
ffi_collectives.h/.cpp: generic helper to request/fetch XLA's communicator from any FFI handler, gated behindXLA_FFI_COLLECTIVES_AVAILABLE(__has_include) since not all jaxlib builds ship the required header.ep.cpp/ep.py: a one-shot bootstrap op fetches the borrowed comm and initializesEPBackend; per-step dispatch/combine ops are unchanged.tex.ep.use_nccl_comm_from_xla()auto-selects the path; switching paths mid-process is rejected with an error.atexitteardown on the borrowed-comm path (XLA may already be tearing down its comm at exit) - useep_finalize()for mid-process teardown there.Testing
TestEPBorrowedCommreruns theTestEPsuite on the borrowed-comm path (smoke subset by default, full suite at L2).TestEpCommSelection: GPU-free unit tests for path-selection gating.qa/L2_jax_distributed_unittest/test.shruns the borrowed-comm suite via the multi-process EP launcher.Checklist: