Repository navigation
Conversation
Paddle-Bot Review Board (review完成)
可维护性建议(总分 85.3 / 100,只给建议,不影响评审结论)
各维度得分(仅评本 PR 改动,点击展开)
Powered by Nyanpasu claude with Opus 5.5 默认推理级别, please check the suggestions carefully. |
| CUDA_CHECK(cudaMemcpyBatchAsync(dst_pointers.data(), | ||
| src_pointers.data(), | ||
| sizes.data(), | ||
| dst_pointers.size(), | ||
| ©_attrs, | ||
| &fail_index, | ||
| 1, | ||
| g_stream_intra_node_copy)); |
There was a problem hiding this comment.
这里和第 889 行都按 CUDA 13.0 的 8 参数签名调用 cudaMemcpyBatchAsync。CUDA 12.8/12.9 的签名有 9 个参数:(void **dsts, void **srcs, size_t *sizes, size_t count, cudaMemcpyAttributes *attrs, size_t *attrsIdxs, size_t numAttrs, size_t *failIdx, cudaStream_t stream),cuda_runtime.h 里的两个模板重载分别是 9 个和 7 个参数。8 个实参匹配不到任何一个声明,src_pointers.data()(const void**)也不能隐式转成 void**。门控 collective/CMakeLists.txt:48 放行的是 CUDA_VERSION >= 12.8,所以在 12.8/12.9 上打开门控时这个文件编译不过。
另外,CUDA 13 下第 6 个实参的位置是 attrsIdxs,这里传的是名为 fail_index 的变量。它的值是 0,结果碰巧正确,但变量名会误导读者。
建议二选一:
- 按
CUDART_VERSION分两种写法。12.x 补上attrsIdxs,并把 srcs 用const_cast转成void**;13.x 保持现在的调用,把变量改名为attrs_idx。 - 把门控提到
CUDA_VERSION >= 13.0,同时修改该处的注释。
| if (!comm_group.send || | ||
| send_offset + (size_t)numel * 4 > comm_group.send_bytes) { |
There was a problem hiding this comment.
numel 是 Python 传进来的 long long。这里的越界检查 send_offset + (size_t)numel * 4 > send_bytes 会因为无符号回绕而失效。例如 numel = -1、send_offset = 8 时,左边回绕成 4,检查通过。随后 launch_scale_range 收到 numel = SIZE_MAX,kernel 会一直写到 send window 之外。blocks_for_numel 里的 (numel + 255) 也会回绕,结果是只用 1 个 block 越界写,最终表现为 illegal address 或静默踩坏显存。gather_put_to_owner 第 873 行的 send_offset + numel * 4 > comm_group.send_bytes 是同样的写法,而且它没有检查 shard_numel < 0。
建议两处都改成不会溢出的形式,例如:
if (numel < 0 || send_offset > send_bytes || (size_t)numel > (send_bytes - send_offset) / 4) return (int)cudaErrorInvalidValue;
| int GradCommEngine::record_window_mark() { | ||
| if (g_window_marks.empty()) return -1; | ||
| cudaStream_t compute_stream = nullptr; | ||
| if (paddle_compute_stream(&compute_stream)) return -1; | ||
| const int i = g_window_mark_index; | ||
| g_window_mark_index = (g_window_mark_index + 1) % (int)g_window_marks.size(); | ||
| if (cudaEventRecord(g_window_marks[i], compute_stream) != cudaSuccess) | ||
| return -1; | ||
| return i; |
There was a problem hiding this comment.
💡 可维护性建议(错误处理一致性,不阻塞合入):GradCommEngine 在头文件里约定返回码为“0 成功,<10000 为 cudaError_t”,但 record_window_mark 成功时返回槽号,失败时一律返回 -1。三种失败(mark 环未配置、取计算流失败、cudaEventRecord 失败)因此无法区分,-1 交给 error_string 也翻译不出原因。建议改成 int record_window_mark(int* slot),槽号走出参,返回值只放真实错误码(未配置时返回 cudaErrorNotReady,取流失败时返回 paddle_compute_stream 的返回码)。
| const int rc = launch_put_to_expert_peers(comm_group.dev_comm, | ||
| comm_group.win_recv, | ||
| comm_group.win_send, | ||
| comm_group.win_host, | ||
| recv_offset, | ||
| send_offset, |
There was a problem hiding this comment.
💡 可维护性建议(API 表面一致性,不阻塞合入):两个结构相同的 put 封装,偏移参数顺序相反。launch_put_to_node_peers 收 (send_offset, recv_offset, ...)(第 540-541 行),这里的 launch_put_to_expert_peers 收 (recv_offset, send_offset, ...)。两个参数都是 size_t,以后改调用时写反了编译器不会报错。建议统一成同一顺序,例如都用 (send_offset, recv_offset, signal_offset),并同步修改 kernels.h 里的声明。
| if (!g_buckets_ready || g_gather_ready || num_slots < 0) | ||
| return (int)cudaErrorNotReady; |
There was a problem hiding this comment.
💡 可维护性建议(逻辑清晰度,不阻塞合入):这里把状态检查和参数校验合在一个条件里,num_slots < 0 也返回 cudaErrorNotReady,报出的是“尚未就绪”,看不出是参数错误。第 869 行 gather_put_to_owner 对非法 group 也返回 NotReady,而同文件其他接口(alloc_windows、scale_range 等)遇到非法 group 都返回 cudaErrorInvalidValue。建议把两处都拆开:状态不满足时返回 NotReady,参数非法时返回 InvalidValue。
| { | ||
| const int rc = launch_signal_intra_arrive(arrive_slots, | ||
| ranks_per_node, | ||
| rank_in_node, | ||
| seq, | ||
| g_stream_intra_node_copy); | ||
| if (rc) return rc; | ||
| } |
There was a problem hiding this comment.
💡 可维护性建议(底层优雅度,不阻塞合入):本文件有十几处同样的 { const int rc = launch_xxx(...); if (rc) return rc; } 匿名块,例如第 453-460、485-496、623-634、636-647、814-821 行。建议在 gin_engine.h 的 CUDA_CHECK / NCCL_CHECK 旁边加一个 GIN_CHECK(x)(do { const int _rc = (x); if (_rc) return _rc; } while (0)),把这些块都换成一行,提交函数的主流程会清楚很多。
| int submit_dense_intra_node_copy(int bucket, | ||
| size_t recv_offset, | ||
| size_t send_offset, | ||
| long long shard_numel, | ||
| cudaEvent_t mark_event, | ||
| size_t intra_arrive_offset, | ||
| unsigned long long seq) { |
There was a problem hiding this comment.
💡 可维护性建议(测试策略,不阻塞合入):这一层新增了六个阶段提交函数和 2D gather,有 offset/numel 越界检查和多条按拓扑分支的路径(ranks_per_node < 2、num_nodes == 1、owner 在同节点或跨节点),但目前没有任何用例覆盖。PR 描述说测试会在后续 PR 补上。建议后续的多进程用例至少覆盖单节点、多节点、owner 同节点和跨节点四种组合,和 reduce_scatter / all_gather 的结果逐位比对,再加一个非法 offset/numel 的入参用例。
c3728a8 to
d44987d
Compare
Paddle links neither libnccl nor libcuda, so new code that calls them has to go through the phi::dynload dlopen wrappers. The gin_overlap gradient communication added in the following commits needs three symbols that are not wrapped yet: * ncclDevCommCreate / ncclGetLsaDevicePointer, the two host entry points of NCCL's device (LSA/GIN) API. They are declared in nccl_device.h, which also carries the device-side inline implementations of the whole device API, so including it from nccl.h would land all of that in every translation unit that reaches for any NCCL symbol. The declarations it needs are repeated here instead, the way the internal entry points at the top of the file already are, and are skipped in translation units that include nccl_device.h themselves. Same 2.30.7 floor as the zero-SM group above them: below that the feature is refused, not degraded. * cuMemcpy2DAsync, for the pitched copy that splits a fused all_gather back into per-bucket storages. It is wrapped under the name libcuda exports, cuMemcpy2DAsync_v2: the wrapper dlsym()s the name it is given but types itself from decltype(&::name), which cuda.h's macro resolves to the v2 declaration, so passing the plain name would call the v1 entry point (whose parameter is the old CUDA_MEMCPY2D_v1 layout) through a v2 signature. Call sites keep writing cuMemcpy2DAsync and let cuda.h's macro rewrite it. The host overload of ncclTeamWorld is deliberately not wrapped: it shares a name with the device-side ncclTeamWorld(ncclDevComm const&), so decltype(&::ncclTeamWorld) cannot be formed, and no host code needs it. Co-Authored-By: Claude Code <noreply@anthropic.com>
ncclDevCommCreate and ncclGetLsaDevicePointer need the same NCCL 2.30.7 as the other symbols of NCCL_RAND_ROUTINE_EACH_AFTER_23007, so they join that group instead of a separate NCCL_RAND_ROUTINE_EACH_DEVICE_API under an identical #if. cuMemcpy2DAsync_v2 has no version gate and joins CUDA_ROUTINE_EACH; CUDA_ROUTINE_EACH_MEMCPY and its two expansions go. The comment on the nccl_device.h prototypes now states what actually holds: the prototypes must match nccl_device.h exactly, because this header is usually included first and both declarations then coexist; the guard only drops the duplicate in the other include order. Co-Authored-By: Claude Code <noreply@anthropic.com>
….30.7 gate Co-Authored-By: Claude Code <noreply@anthropic.com>
d44987d to
c61dbf0
Compare
c61dbf0 to
fd3f49c
Compare
First piece of moving the NCCL-GIN sharding-stage1 gradient primitives out of PaddleFleet's extension (paddlefleet_ops/gin_overlap/csrc) into Paddle: the CUDA kernels and their launch wrappers, plus the CMake gate that decides whether the directory is built at all. Nothing calls these yet; the host-side engine that launches them comes in the next commits. * kernels/reduce.cu: the intra-node column sum, the dense shard sum across nodes and the expert shard sum, each scaled in place. * kernels/put.cu: GIN one-sided puts to the same-rank peers on other nodes, to the expert-group peers, to the 2D-bucket owner, and the GIN flush. * kernels/signal.cu: the intra-node counter / arrival writes over LSA pointers, the GIN credit signal, a single release store, and the lookup of the host-window LSA pointers. * kernels/scale.cu: an in-place scale of a send-window range. * kernels/kernels.h: the launch_* / preload_* declarations the host side is allowed to call (no <<<>>> outside kernels/). * gin_dynload.h: brings the phi::dynload wrappers (previous commit) into namespace gin_overlap under their plain names, so the host code that comes next calls NCCL / the driver through dlopen without touching call sites. The code is moved, not rewritten: the sum order, the kernels and the block counts are untouched; the code-level diff against the extension is include paths, namespaces and include guards (#pragma once). Two casts of CUDA built-ins are respelled as static_cast so that the no-int64-type-cuda-index-without-cast ast-grep rule sees them; cpplint is switched off for the directory, as it is for deep_ep. The comments say what each kernel does and what must hold; reduce.cu's header lists what keeps the result bit-identical to reduce_scatter (ring order starting at own rank + 1, __fmul_rn before __fadd_rn) and states that the block / thread count is not among them: it only decides which indices a thread walks. The caller does change the reduce kernels' block count between windows within one run, and the bit-exactness check against reduce_scatter passes with that. Build gate (CMake is written but not run in this change): * arch: only built when NVCC_ARCH_BIN contains 9.0 or newer, computed the way paddle/phi/kernels/CMakeLists.txt computes has_arch_ge80. Inside the directory the <= 8.x gencodes are stripped off CMAKE_CUDA_FLAGS, as deep_ep/CMakeLists.txt does. * NCCL >= 2.30.7 with nccl_device.h present, and CUDA >= 12.8 for cudaMemcpyBatchAsync. The result is cached as WITH_GIN_OVERLAP for the pybind layer added later. gin_dynload.h #errors below the NCCL floor as a backstop for the gate. Co-Authored-By: Claude Code <noreply@anthropic.com>
Translate every comment, docstring, error message and log line of the gin_overlap sources (C++, CUDA, CMake, pybind and the Python layer) into English, and tighten the wording on the way. The terms are defined once, in the gin_engine.h file header (C++) and in the engine.py module docstring (Python); every other file uses them as defined there. Strings only. comment_clean_check.py --ignore-strings passes on every changed file: C++ / CUDA / CMake token streams are identical except for string literals (printf conversions inside merged literals keep their order), and the Python ASTs are identical except for str constants and the literal parts of f-strings. Log lines all carry the [gin_overlap] prefix; copy_2d errors now start with "copy_2d:". Co-Authored-By: Claude Code <noreply@anthropic.com>
…the gate Both cudaMemcpyBatchAsync calls use the CUDA 13 signature (8 arguments, const void** sources). CUDA 12.8 / 12.9 declare a 9-argument version, so a build with the gate open on those toolkits would not compile. Raise the gate to CUDA >= 13.0, the only toolkit the code has been built and run with, instead of adding an untested 12.x branch; gin_dynload.h gets the matching #error backstop. The batch attribute index is named attrs_idx, which is what that argument is. The arch condition reuses ARCH_BIN_CONTAINS_GE_90, computed at the top of the same file for DeepEP, instead of a second loop over NVCC_ARCH_BIN. _BUILD_HINT in the Python layer now lists exactly the gate's conditions, and the CMake comment says to change both together. Co-Authored-By: Claude Code <noreply@anthropic.com>
…ne API
Argument checks (values come from Python):
- scale_range, gather_put_to_owner, send_window_tensor and copy_2d check
their bounds without unsigned wrap-around, and reject numel < 0.
copy_2d also rejects non-contiguous tensors (a stride-0 or transposed
view passes a numel-based bound but the copy reads contiguous bytes).
- release::setup checks every offset of the per-bucket tables and the
host window areas: non-negative, aligned (counters 8 B, data 4 B) and
inside its window. setup_gather_table checks both arrival counter
offsets of each entry the same way. CommGroup records recv_bytes for
this, and alloc_symmetric_window returns the size it actually
registered instead of the caller repeating the 4096 B minimum.
- A wrong state returns cudaErrorNotReady and a bad argument
cudaErrorInvalidValue (gather_setup, gather_put_to_owner, scale_range).
- bucket_ready rejects a bucket reported twice in a step in the RUN state
too, from a bitmap only the main thread touches.
Init failures:
- configure destroys the streams and events it created, frees its
temporary buffer and resets the globals it set when a later step fails,
so it can be retried. set_comms clears the groups it set on failure.
release::setup unpublishes its control block if the tail thread cannot
be started. The steps that cannot be rolled back (window registration,
devcomm creation, the host window) say so: restart the process.
- The gin_engine.h constraints state that the engine lives as long as
the process; shutdown_process_group / restart_process_group are not
supported.
API:
- GIN_CHECK replaces the dozen { rc = launch_x(...); if (rc) return rc; }
blocks. launch_* / preload_* return cudaError_t.
- launch_put_to_expert_peers takes (send_offset, recv_offset) like
launch_put_to_node_peers.
- join_bucket_streams / gather_wait_bucket take kind (0 dense, 1 expert),
as bucket_ready and the gather table do, instead of is_dense.
- The six release positions are named variables, one per stage.
- The methods that do not return a code (record_window_mark, tail_mark,
bucket_ready, tail_all_posted, tail_wait, tail_abort, release_activity,
the version getters) document what they return.
- tail_wait / tail_abort release the GIL while they block.
No kernel, reduction order, release condition or credit target changes.
Co-Authored-By: Claude Code <noreply@anthropic.com>
kernels/release.cu has no kernel and no <<<>>>: it is the host-side release state machine and its std::thread. kernels/ holds only the kernel launch wrappers, so the file moves next to gin_engine.cc as release.cc and is compiled by the host compiler instead of nvcc. The content is unchanged apart from the file name in comments. Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Remove gin_overlap/ from the cpplint exclude list and fix every finding: C-style casts become static_cast / reinterpret_cast, long long and unsigned long long become int64_t / uint64_t, non-const reference out-parameters become pointers (call sites updated), and C system headers come before C++ ones. atomicAdd_system keeps its unsigned long long prototype through one aliased type. No change to values, types seen by Python, or arithmetic. Co-Authored-By: Claude Code <noreply@anthropic.com>
…+ sources only The four warning options are wrapped in $<$<COMPILE_LANGUAGE:CXX>:...>, so nvcc no longer receives them for kernels/*.cu. Co-Authored-By: Claude Code <noreply@anthropic.com>
…treams Second piece of the move from PaddleFleet's extension: the host side that builds everything the per-step code runs on. Nothing calls it yet (the Python binding comes last), and nothing is submitted to the GPU per step yet (next commit). * gin_engine.h / gin_engine.cc, initialization only: the CommGroup state of the two communication groups (sharding / moe_sharding), the error-check macros, and the GradCommEngine methods that the Python layer calls once at startup, in this order: set_comms (reuse Paddle's ncclComm_t for each ring id and cross-check size / rank), alloc_windows (send / recv / signal as NCCL_WIN_COLL_SYMMETRIC windows), alloc_host_counters (one HOST_NUMA host window per group, registered with NCCL_WIN_STRICT_ORDERING), create_dev_comm (ncclDevCommCreate + LSA pointers), set_group_topology, and configure (side streams, per-bucket events, the window-mark ring, the credit offsets, the host-window LSA pointers, and preloading every kernel so that the first launch of a module does not land on the backward critical path). Also error_string and the NCCL runtime / build version queries. * paddle_comm.cc: takes the ncclComm_t out of Paddle's ProcessGroup by ring id, the way DeepEP does (ProcessGroupMapFromGid -> ProcessGroupNCCL -> NCCLCommContext::GetNcclComm). * paddle_tensor.cc / .h: Paddle's current compute stream, the zero-copy paddle::Tensor over a send-window slice (paddle::from_blob with deleter = nullptr, so the memory stays owned by the NCCL window, plus a post-wrap self-check), and the pitched copy_2d through cuMemcpy2DAsync. The file header of gin_engine.h is the one full description of the C++ side (terms, the two groups, the dense / expert / 2D stages, the receive pools, the host-window layout and the constraints). It is added here in full: it is organised by concept, not by layer, so the parts about the per-stage submits and the release layer (next two commits) cannot be moved out without rewriting it. paddle_comm.cu and paddle_tensor.cu become .cc: they were .cu only because an out-of-tree build has to hand any translation unit that includes Paddle headers to nvcc (thrust); in-tree they are ordinary sources. The pybind registration that was stranded in paddle_tensor.cu moves to the pybind layer, so paddle_tensor.h declares the real paddle::Tensor signatures. The library now also depends on process_group_nccl, for paddle_comm.cc. Like the kernels, the code is moved, not rewritten: same checks, same order of collective calls. Co-Authored-By: Claude Code <noreply@anthropic.com>
Translate every comment, docstring, error message and log line of the gin_overlap sources (C++, CUDA, CMake, pybind and the Python layer) into English, and tighten the wording on the way. The terms are defined once, in the gin_engine.h file header (C++) and in the engine.py module docstring (Python); every other file uses them as defined there. Strings only. comment_clean_check.py --ignore-strings passes on every changed file: C++ / CUDA / CMake token streams are identical except for string literals (printf conversions inside merged literals keep their order), and the Python ASTs are identical except for str constants and the literal parts of f-strings. Log lines all carry the [gin_overlap] prefix; copy_2d errors now start with "copy_2d:". Co-Authored-By: Claude Code <noreply@anthropic.com>
…ne API
Argument checks (values come from Python):
- scale_range, gather_put_to_owner, send_window_tensor and copy_2d check
their bounds without unsigned wrap-around, and reject numel < 0.
copy_2d also rejects non-contiguous tensors (a stride-0 or transposed
view passes a numel-based bound but the copy reads contiguous bytes).
- release::setup checks every offset of the per-bucket tables and the
host window areas: non-negative, aligned (counters 8 B, data 4 B) and
inside its window. setup_gather_table checks both arrival counter
offsets of each entry the same way. CommGroup records recv_bytes for
this, and alloc_symmetric_window returns the size it actually
registered instead of the caller repeating the 4096 B minimum.
- A wrong state returns cudaErrorNotReady and a bad argument
cudaErrorInvalidValue (gather_setup, gather_put_to_owner, scale_range).
- bucket_ready rejects a bucket reported twice in a step in the RUN state
too, from a bitmap only the main thread touches.
Init failures:
- configure destroys the streams and events it created, frees its
temporary buffer and resets the globals it set when a later step fails,
so it can be retried. set_comms clears the groups it set on failure.
release::setup unpublishes its control block if the tail thread cannot
be started. The steps that cannot be rolled back (window registration,
devcomm creation, the host window) say so: restart the process.
- The gin_engine.h constraints state that the engine lives as long as
the process; shutdown_process_group / restart_process_group are not
supported.
API:
- GIN_CHECK replaces the dozen { rc = launch_x(...); if (rc) return rc; }
blocks. launch_* / preload_* return cudaError_t.
- launch_put_to_expert_peers takes (send_offset, recv_offset) like
launch_put_to_node_peers.
- join_bucket_streams / gather_wait_bucket take kind (0 dense, 1 expert),
as bucket_ready and the gather table do, instead of is_dense.
- The six release positions are named variables, one per stage.
- The methods that do not return a code (record_window_mark, tail_mark,
bucket_ready, tail_all_posted, tail_wait, tail_abort, release_activity,
the version getters) document what they return.
- tail_wait / tail_abort release the GIL while they block.
No kernel, reduction order, release condition or credit target changes.
Co-Authored-By: Claude Code <noreply@anthropic.com>
kernels/release.cu has no kernel and no <<<>>>: it is the host-side release state machine and its std::thread. kernels/ holds only the kernel launch wrappers, so the file moves next to gin_engine.cc as release.cc and is compiled by the host compiler instead of nvcc. The content is unchanged apart from the file name in comments. Co-Authored-By: Claude Code <noreply@anthropic.com>
Remove gin_overlap/ from the cpplint exclude list and fix every finding: C-style casts become static_cast / reinterpret_cast, long long and unsigned long long become int64_t / uint64_t, non-const reference out-parameters become pointers (call sites updated), and C system headers come before C++ ones. atomicAdd_system keeps its unsigned long long prototype through one aliased type. No change to values, types seen by Python, or arithmetic. Co-Authored-By: Claude Code <noreply@anthropic.com>
…e slot as an out-parameter C++: both return 0 or a cudaError_t code and write the slot to *slot: cudaErrorNotReady without configure / release_setup, the error of paddle_compute_stream or cudaEventRecord, and cudaErrorLaunchOutOfResources when this step's tail mark slots are used up (was -1 / -2). The Python binding still returns the slot and raises RuntimeError with the error text on failure. CPU test of the binding before setup. Co-Authored-By: Claude Code <noreply@anthropic.com>
…ther Third piece of the move from PaddleFleet's extension: the host functions that hand one stage of one bucket to its side stream, and the stream / event plumbing around them. Still no caller: the release layer that decides when each stage may go (next commit) is the only code that calls submit_*. * The six stage submits: submit_dense_intra_node_copy (cudaMemcpyBatchAsync of the column to the other ranks on the node + the arrival write), submit_dense_intra_node_reduce (+ intra-node credit), submit_dense_inter_node_put, submit_moe_put, submit_dense_inter_node_reduce and submit_moe_reduce (+ GIN credit). Each one only waits on work this rank has already handed out (a compute-stream mark or a local side-stream event), never on a remote counter: whether the remote side is ready is decided on the host from the host window before the call. * record_window_mark: records a mark on Paddle's current compute stream into the ring configure() created. * join_bucket_streams (compute stream waits for a bucket's side-stream events), scale_range, end_step (GIN flush of both groups so the send window can be reused), stats. * Muon 2D-bucket gather to the owner: gather_setup, gather_gate, gather_wait_bucket, gather_mark_ready, gather_put_to_owner (GIN put across nodes, CE + release store within a node) and gather_join. The table-driven entry point, gather_put_entry, forwards to the release layer and comes with it. Moved as is from the extension, like the previous commits. Co-Authored-By: Claude Code <noreply@anthropic.com>
Translate every comment, docstring, error message and log line of the gin_overlap sources (C++, CUDA, CMake, pybind and the Python layer) into English, and tighten the wording on the way. The terms are defined once, in the gin_engine.h file header (C++) and in the engine.py module docstring (Python); every other file uses them as defined there. Strings only. comment_clean_check.py --ignore-strings passes on every changed file: C++ / CUDA / CMake token streams are identical except for string literals (printf conversions inside merged literals keep their order), and the Python ASTs are identical except for str constants and the literal parts of f-strings. Log lines all carry the [gin_overlap] prefix; copy_2d errors now start with "copy_2d:". Co-Authored-By: Claude Code <noreply@anthropic.com>
…the gate Both cudaMemcpyBatchAsync calls use the CUDA 13 signature (8 arguments, const void** sources). CUDA 12.8 / 12.9 declare a 9-argument version, so a build with the gate open on those toolkits would not compile. Raise the gate to CUDA >= 13.0, the only toolkit the code has been built and run with, instead of adding an untested 12.x branch; gin_dynload.h gets the matching #error backstop. The batch attribute index is named attrs_idx, which is what that argument is. The arch condition reuses ARCH_BIN_CONTAINS_GE_90, computed at the top of the same file for DeepEP, instead of a second loop over NVCC_ARCH_BIN. _BUILD_HINT in the Python layer now lists exactly the gate's conditions, and the CMake comment says to change both together. Co-Authored-By: Claude Code <noreply@anthropic.com>
…ne API
Argument checks (values come from Python):
- scale_range, gather_put_to_owner, send_window_tensor and copy_2d check
their bounds without unsigned wrap-around, and reject numel < 0.
copy_2d also rejects non-contiguous tensors (a stride-0 or transposed
view passes a numel-based bound but the copy reads contiguous bytes).
- release::setup checks every offset of the per-bucket tables and the
host window areas: non-negative, aligned (counters 8 B, data 4 B) and
inside its window. setup_gather_table checks both arrival counter
offsets of each entry the same way. CommGroup records recv_bytes for
this, and alloc_symmetric_window returns the size it actually
registered instead of the caller repeating the 4096 B minimum.
- A wrong state returns cudaErrorNotReady and a bad argument
cudaErrorInvalidValue (gather_setup, gather_put_to_owner, scale_range).
- bucket_ready rejects a bucket reported twice in a step in the RUN state
too, from a bitmap only the main thread touches.
Init failures:
- configure destroys the streams and events it created, frees its
temporary buffer and resets the globals it set when a later step fails,
so it can be retried. set_comms clears the groups it set on failure.
release::setup unpublishes its control block if the tail thread cannot
be started. The steps that cannot be rolled back (window registration,
devcomm creation, the host window) say so: restart the process.
- The gin_engine.h constraints state that the engine lives as long as
the process; shutdown_process_group / restart_process_group are not
supported.
API:
- GIN_CHECK replaces the dozen { rc = launch_x(...); if (rc) return rc; }
blocks. launch_* / preload_* return cudaError_t.
- launch_put_to_expert_peers takes (send_offset, recv_offset) like
launch_put_to_node_peers.
- join_bucket_streams / gather_wait_bucket take kind (0 dense, 1 expert),
as bucket_ready and the gather table do, instead of is_dense.
- The six release positions are named variables, one per stage.
- The methods that do not return a code (record_window_mark, tail_mark,
bucket_ready, tail_all_posted, tail_wait, tail_abort, release_activity,
the version getters) document what they return.
- tail_wait / tail_abort release the GIL while they block.
No kernel, reduction order, release condition or credit target changes.
Co-Authored-By: Claude Code <noreply@anthropic.com>
kernels/release.cu has no kernel and no <<<>>>: it is the host-side release state machine and its std::thread. kernels/ holds only the kernel launch wrappers, so the file moves next to gin_engine.cc as release.cc and is compiled by the host compiler instead of nvcc. The content is unchanged apart from the file name in comments. Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Remove gin_overlap/ from the cpplint exclude list and fix every finding: C-style casts become static_cast / reinterpret_cast, long long and unsigned long long become int64_t / uint64_t, non-const reference out-parameters become pointers (call sites updated), and C system headers come before C++ ones. atomicAdd_system keeps its unsigned long long prototype through one aliased type. No change to values, types seen by Python, or arithmetic. Co-Authored-By: Claude Code <noreply@anthropic.com>
…e slot as an out-parameter C++: both return 0 or a cudaError_t code and write the slot to *slot: cudaErrorNotReady without configure / release_setup, the error of paddle_compute_stream or cudaEventRecord, and cudaErrorLaunchOutOfResources when this step's tail mark slots are used up (was -1 / -2). The Python binding still returns the slot and raises RuntimeError with the error text on failure. CPU test of the binding before setup. Co-Authored-By: Claude Code <noreply@anthropic.com>
fd3f49c to
3406e43
Compare
PR Category
Distributed Strategy
PR Types
New features
Description
Merged into #79970.
This PR is stacked on #79968: Files changed also shows the earlier layers. Changes of this layer only: AlAuAu/Paddle@gin_split_3_engine_init...gin_split_4_engine_submit
Part 4/6 of the gin_overlap primitive (#79964), split for review. Compiled with the gate on (CUDA 13.2, NCCL 2.31.2,
CUDA_ARCH_BIN="90 100 103"); not run on a GPU yet.The six PRs of this series: #79966 → #79967 → #79968 → #79969 → #79970 → #79971. Each branch contains the ones before it, so on GitHub every PR after the first shows the earlier layers too; review only the commits of this layer (the Commits tab), and merge in order.
Series (each PR builds on the previous one; the six together equal #79964):
1 dynload wrappers → 2 kernels + build gate → 3 engine setup → 4 per-stage submits (this PR) →
5 release layer → 6 pybind + Python layer.
What this PR adds (
gin_engine.h/gin_engine.cconly)The host functions that hand one stage of one bucket to its side stream, and the stream / event
plumbing around them.
submit_dense_intra_node_copy(cudaMemcpyBatchAsyncof the column to theother ranks on the node + the arrival write),
submit_dense_intra_node_reduce(+ intra-nodecredit),
submit_dense_inter_node_put,submit_moe_put,submit_dense_inter_node_reduceandsubmit_moe_reduce(+ GIN credit). Each waits only on work this rank has already handed out (acompute-stream mark or a local side-stream event), never on a remote counter.
record_window_mark: records a mark on Paddle's current compute stream into the ringconfigure()created;GIN_COMPUTE_STREAM, the macro that fetches that stream.join_bucket_streams(compute stream waits for a bucket's side-stream events),scale_range,end_step(GIN flush of both groups so the send window can be reused),stats.gather_setup,gather_gate,gather_wait_bucket,gather_mark_ready,gather_put_to_owner(GIN put across nodes, CE + release store within anode),
gather_join. The table-driven entry pointgather_put_entryforwards to the releaselayer and comes with it in [WIP][Distributed] gin_overlap 2/3: engine and release layer (C++) #79970.
Depends on: #79968 (
gin_split_3_engine_init).Callers after merge: none. The only caller of
submit_*is the release layer (#79970); theGradCommEnginemethods are called from Python (#79971).What CI can show: on a build with the gate on (see #79967), that the two files compile. No test
calls them.
How correctness is established: moved from PaddleFleet's extension unchanged. The bitwise
evidence so far is for earlier, out-of-tree states of the engine (4 nodes / 32 cards, 10 steps,
identical to Paddle's
reduce_scatterbaseline; see #79967), not for this tree. The acceptancegate is the same 10-step AdamW and Muon run on a build of the whole series. The whole
series has been compiled (see #79967); nothing has been run on a GPU yet.
Changes after the first review
attrs_idx.scale_rangeandgather_put_to_ownercheck their bounds without wrap-around and rejectnumel < 0.cudaErrorNotReadyand a bad argumentcudaErrorInvalidValue(gather_setup,gather_put_to_owner,scale_range).join_bucket_streams/gather_wait_buckettakekind(0 dense, 1 expert), asbucket_readyand the gather table do, instead ofis_dense.record_window_marknow returns an error code.)record_window_markkeeps returning the slot; its declaration now says so (an out-parameter would change the Python binding for no behavior change).reduce_scatter). They need GPUs and come with the acceptance run.Not verified: none of these changes has run on a GPU. Every change is outside the numerical path (no kernel body, summation order,
__fmul_rn/__fadd_rnorder, release condition, credit target or coefficient placement changed); the 4-node bitwise acceptance run is still to come. Comments, docstrings, errors and logs of the series are now in English (one commit per layer; outside string literals the token stream / AST is identical, checked mechanically).Changes after the second review
record_window_marknow returns an error code.)record_window_mark's declaration ingin_engine.hlists its return values: the window mark slot (>= 0), or -1 whenconfigurehas not run, the compute stream cannot be read, orcudaEventRecordfailed (same style asbucket_ready/tail_mark). Comment only, no behavior change.int64_t/uint64_t, pointer out-parameters, include order); details in [WIP][Distributed] gin_overlap 1/3: dynload wrappers, device kernels and the build gate #79967.record_window_mark(int* slot)returns an error code and writes the slot to*slot(was: the slot, or -1 for every failure):cudaErrorNotReadywithoutconfigure, the error ofpaddle_compute_stream, or thecudaError_tofcudaEventRecord. The declaration says so.Not verified: the precision acceptance run (4 nodes, bitwise) is still in progress, and the changes of this round have not been through it. The behavior changes of this round are the argument check in
release::setup(#79970) and the error codes ofrecord_window_mark/tail_mark(#79969 / #79970, Python still gets the slot); the cpplint fixes (#79967) change casts, integer type spellings and reference -> pointer parameters only.是否引起精度变化
否
🤖 Generated with Claude Code