Advance Corgi timestamps in runtime-width rows - #872
Draft
frankmcsherry wants to merge 1 commit into
Draft
frankmcsherry wants to merge 1 commit into
frankmcsherry wants to merge 1 commit into
Conversation
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Corgi trace compaction reconstructs an owned dynamic timestamp per row to advance and sort it. This adds a buffer-level kernel for DDIR's numeric product timestamps, using one flat buffer of runtime-width rows.
The width includes the outer epoch and covers the complete input prefix plus the whole frontier; missing coordinates are zero-padded. Numeric min/max distributivity lets the kernel compute frontier coordinate minima once, then advance the rows in bulk. It sorts/consolidates row indices within complete key/value groups and writes canonical coordinates directly back to the timestamp columns. Empty frontiers leave times unchanged. Other timestamp types retain the generic lattice path.
The change is confined to timestamp-column helpers and chunk compaction: +143/-3 production/documentation lines, plus 182 test/wiring lines. Width is a runtime value; there is no prepared arity family.
Fresh maintenance measurements against
master-nextat229508dd(ms/update):Apple M4, one worker, mimalloc, release/LTO, locked dependencies; four fresh processes, median of process means over updates 6-25. Reach uses 200k nodes/400k edges; SCC 100k/200k; 1,000 replaced edges/update. Measured with a local in-process graph harness, excluding input preparation. Low-churn, sparse and dense graph screens also improved for both seeds.
Memory trade-off: scratch is rows times runtime width. Retained requested heap was unchanged. Peak requested heap increased 7.4 MiB on Reach and was unchanged on SCC; observed peak RSS increased about 22 MiB on SCC. SCC allocation calls fell from 14.66M to 10.83M/update.
Validation: workspace/all-target tests, doctests, clippy, release explanation tests including ignored soundness sweeps, one/four-worker LDBC and SNB checks, and 522 independent graph-oracle snapshots. New tests cover deep/incomparable frontiers, canonical output, shared chunks, cancellation, incomplete groups, and a group larger than the output chunk target.