Declare and instrument PCIe H2D and D2H byte counter and per-device label telemetry metrics: - #950
Open
copybara-service[bot] wants to merge 1 commit into
Open
copybara-service[bot] wants to merge 1 commit into
copybara-service[bot] wants to merge 1 commit into
Conversation
copybara-service
Bot
force-pushed
the
test_982457553
branch
from
September 20, 2026 19:07
2486c41 to
09216d7
Compare
copybara-service
Bot
changed the title
…abel telemetry metrics:
- Declare `kH2dBytesTotal` (`h2d_bytes_total`), `kD2hBytesTotal` (`d2h_bytes_total`), and `kHostIp` (`host_ip`) in `metrics_backend.h`, and register `kH2dBytesTotal` and `kD2hBytesTotal` in `kAllMetrics` (`metrics_backend.h`) with `kPcieTransferLabels` (`{host_ip, local_rank}`), while keeping `kH2dTransferTimeMs` and `kD2hTransferTimeMs` unlabeled with unchanged emission behavior.
- Add anonymous-namespace helpers `ComputeBytesPerShard` (computing per-shard byte counts across layers from `copy_sizes_major_dim` for partial block transfers or full-layer `physical_size` / `max_physical_size_` for full-buffer transfers), `GetShardRankString`, and `JoinAndRecordTelemetry` in `kv_cache_manager_base.cc` to increment `h2d_bytes_total` and `d2h_bytes_total` per shard upon successful transfer completion across `H2dSyncDispatch`, `D2hSyncDispatch`, `D2hWrite`, `H2dDirect`, `D2hDirect`, and `CopyPoolBlocks`.
- Add unit tests in `kv_cache_manager_test.cc` and `kv_cache_manager_with_transfer_test.cc` verifying per-device H2D/D2H byte counter emission and partial block byte accounting.
PiperOrigin-RevId: 982457553
copybara-service
Bot
force-pushed
the
test_982457553
branch
from
September 21, 2026 06:56
09216d7 to
0be3052
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Declare and instrument PCIe H2D and D2H byte counter and per-device label telemetry metrics:
kH2dBytesTotal(h2d_bytes_total),kD2hBytesTotal(d2h_bytes_total), andkHostIp(host_ip) inmetrics_backend.h, and registerkH2dBytesTotalandkD2hBytesTotalinkAllMetrics(metrics_backend.h) withkPcieTransferLabels({host_ip, local_rank}), while keepingkH2dTransferTimeMsandkD2hTransferTimeMsunlabeled with unchanged emission behavior.ComputeBytesPerShard(computing per-shard byte counts across layers fromcopy_sizes_major_dimfor partial block transfers or full-layerphysical_size/max_physical_size_for full-buffer transfers),GetShardRankString, andJoinAndRecordTelemetryinkv_cache_manager_base.ccto incrementh2d_bytes_totalandd2h_bytes_totalper shard upon successful transfer completion acrossH2dSyncDispatch,D2hSyncDispatch,D2hWrite,H2dDirect,D2hDirect, andCopyPoolBlocks.kv_cache_manager_test.ccandkv_cache_manager_with_transfer_test.ccverifying per-device H2D/D2H byte counter emission and partial block byte accounting.