Reimplement gridDisk internals with BFS by isaacbrodsky · Pull Request #1217 · uber/h3

isaacbrodsky · 2026-08-09T23:33:06Z

after

% make benchmarks
[ 18%] Built target h3
[ 27%] Built target benchmarkPolygon
        -- pointInsideGeoLoopSmall: 0.016270 microseconds per iteration (100000 iterations)
        -- pointInsideGeoLoopLarge: 0.194830 microseconds per iteration (100000 iterations)
        -- bboxFromGeoLoopSmall: 0.011700 microseconds per iteration (100000 iterations)
        -- bboxFromGeoLoopLarge: 0.167150 microseconds per iteration (100000 iterations)
[ 27%] Built target bench_benchmarkPolygon
[ 36%] Built target benchmarkH3Api
        -- latLngToCell: 0.254400 microseconds per iteration (10000 iterations)
        -- cellToLatLng: 0.125100 microseconds per iteration (10000 iterations)
        -- cellToBoundary: 0.434800 microseconds per iteration (10000 iterations)
[ 36%] Built target bench_benchmarkH3Api
[ 45%] Built target benchmarkGridDiskCells
        -- gridDisk10: 4.235200 microseconds per iteration (10000 iterations)
        -- gridDisk20: 10.766900 microseconds per iteration (10000 iterations)
        -- gridDisk30: 22.694000 microseconds per iteration (10000 iterations)
        -- gridDisk40: 39.115200 microseconds per iteration (10000 iterations)
        -- gridDiskPentagon10: 14.946000 microseconds per iteration (500 iterations)
        -- gridDiskPentagon20: 63.648000 microseconds per iteration (500 iterations)
        -- gridDiskPentagon30: 146.800000 microseconds per iteration (50 iterations)
        -- gridDiskPentagon40: 274.300000 microseconds per iteration (10 iterations)
[ 45%] Built target bench_benchmarkGridDiskCells
[ 45%] Built target benchmarkGridPathCells
        -- gridPathCellsNear: 4.586700 microseconds per iteration (10000 iterations)
        -- gridPathCellsFar: 170.752000 microseconds per iteration (1000 iterations)
[ 45%] Built target bench_benchmarkGridPathCells
[ 45%] Built target benchmarkDirectedEdge
        -- directedEdgeToBoundary: 1.416600 microseconds per iteration (10000 iterations)
        -- reverseDirectedEdge: 0.266100 microseconds per iteration (10000 iterations)
[ 45%] Built target bench_benchmarkDirectedEdge
[ 54%] Built target benchmarkGosperIter
        -- hex_plus0: 0.010582 microseconds per iteration (50000000 iterations)
        -- pent_plus0: 0.008782 microseconds per iteration (50000000 iterations)
        -- hex_plus1: 0.100846 microseconds per iteration (500000 iterations)
        -- pent_plus1: 0.090218 microseconds per iteration (500000 iterations)
        -- hex_plus2: 0.363330 microseconds per iteration (100000 iterations)
        -- pent_plus2: 0.348480 microseconds per iteration (100000 iterations)
        -- hex_plus5: 11.580000 microseconds per iteration (1000 iterations)
        -- pent_plus5: 11.479000 microseconds per iteration (1000 iterations)
        -- hex_plus8: 312.900000 microseconds per iteration (100 iterations)
        -- pent_plus8: 309.840000 microseconds per iteration (100 iterations)
        -- hex_plus11: 8472.000000 microseconds per iteration (10 iterations)
        -- pent_plus11: 8313.300000 microseconds per iteration (10 iterations)
[ 54%] Built target bench_benchmarkGosperIter
[ 63%] Built target benchmarkVertex
        -- cellToVertexes: 0.860800 microseconds per iteration (10000 iterations)
        -- cellToVertexesPent: 0.040200 microseconds per iteration (10000 iterations)
        -- cellToVertexesRing: 9.336800 microseconds per iteration (10000 iterations)
        -- cellToVertexesRingPent: 9.816900 microseconds per iteration (10000 iterations)
[ 63%] Built target bench_benchmarkVertex
[ 72%] Built target benchmarkIsValidCell
        -- pentagonChildren_2_8: 147.408000 microseconds per iteration (1000 iterations)
        -- pentagonChildren_8_14: 130.829000 microseconds per iteration (1000 iterations)
        -- pentagonChildren_8_14_null_2: 129.006000 microseconds per iteration (1000 iterations)
        -- pentagonChildren_8_14_null_10: 129.448000 microseconds per iteration (1000 iterations)
        -- pentagonChildren_8_14_null_100: 134.888000 microseconds per iteration (1000 iterations)
[ 72%] Built target bench_benchmarkIsValidCell
[ 81%] Built target benchmarkCellsToPolyAlgos
        -- linked_disk2: 11.077300 microseconds per iteration (10000 iterations)
        -- direct_disk2: 8.751500 microseconds per iteration (10000 iterations)
        -- linked_donut: 5.012900 microseconds per iteration (10000 iterations)
        -- direct_donut: 4.688400 microseconds per iteration (10000 iterations)
        -- linked_nestedDonuts: 19.310500 microseconds per iteration (10000 iterations)
        -- direct_nestedDonuts: 18.239000 microseconds per iteration (10000 iterations)
        -- linked_manyChildren: 3047.300000 microseconds per iteration (10 iterations)
        -- direct_manyChildren: 2782.800000 microseconds per iteration (10 iterations)
        -- linked_colorado: 1504.810000 microseconds per iteration (100 iterations)
        -- direct_colorado: 1467.890000 microseconds per iteration (100 iterations)
[ 81%] Built target bench_benchmarkCellsToPolyAlgos
[ 90%] Built target benchmarkCellToChildren
        -- cellToChildren1: 0.023700 microseconds per iteration (10000 iterations)
        -- cellToChildren2: 0.254100 microseconds per iteration (10000 iterations)
        -- cellToChildren3: 1.540600 microseconds per iteration (10000 iterations)
        -- cellToChildren4: 7.867800 microseconds per iteration (10000 iterations)
        -- cellToChildren5: 48.937000 microseconds per iteration (10000 iterations)
[ 90%] Built target bench_benchmarkCellToChildren
[ 90%] Built target benchmarkPolygonToCells
        -- polygonToCellsSF: 349.760000 microseconds per iteration (500 iterations)
        -- polygonToCellsAlameda: 490.524000 microseconds per iteration (500 iterations)
        -- polygonToCellsSouthernExpansion: 15176.000000 microseconds per iteration (10 iterations)
[ 90%] Built target bench_benchmarkPolygonToCells
[100%] Built target benchmarkPolygonToCellsExperimental
        -- polygonToCellsSF_Center: 242.160000 microseconds per iteration (500 iterations)
        -- polygonToCellsSF_Full: 507.324000 microseconds per iteration (500 iterations)
        -- polygonToCellsSF_Overlapping: 618.012000 microseconds per iteration (500 iterations)
        -- polygonToCellsAlameda_Center: 448.580000 microseconds per iteration (500 iterations)
        -- polygonToCellsAlameda_Full: 1044.254000 microseconds per iteration (500 iterations)
        -- polygonToCellsAlameda_Overlapping: 1418.072000 microseconds per iteration (500 iterations)
        -- polygonToCellsSouthernExpansion_Center: 8813.300000 microseconds per iteration (10 iterations)
        -- polygonToCellsSouthernExpansion_Full: 24354.600000 microseconds per iteration (10 iterations)
        -- polygonToCellsSouthernExpansion_Overlapping: 38903.700000 microseconds per iteration (10 iterations)
[100%] Built target bench_benchmarkPolygonToCellsExperimental
[100%] Built target benchmarkArea
res: 0, diff: 3.552714e-15
res: 1, diff: 5.329071e-15
res: 2, diff: 5.329071e-15
res: 3, diff: 1.776357e-15
        -- allCellsAtRes_print: 35598.000000 microseconds per iteration (1 iterations)
        -- allCellsAtRes_noprint: 19896.700000 microseconds per iteration (10 iterations)
[100%] Built target bench_benchmarkArea
[100%] Built target benchmarks

before

% make benchmarks
[ 18%] Built target h3
[ 27%] Built target benchmarkPolygon
        -- pointInsideGeoLoopSmall: 0.020940 microseconds per iteration (100000 iterations)
        -- pointInsideGeoLoopLarge: 0.213100 microseconds per iteration (100000 iterations)
        -- bboxFromGeoLoopSmall: 0.011690 microseconds per iteration (100000 iterations)
        -- bboxFromGeoLoopLarge: 0.173530 microseconds per iteration (100000 iterations)
[ 27%] Built target bench_benchmarkPolygon
[ 36%] Built target benchmarkH3Api
        -- latLngToCell: 0.253600 microseconds per iteration (10000 iterations)
        -- cellToLatLng: 0.124500 microseconds per iteration (10000 iterations)
        -- cellToBoundary: 0.425800 microseconds per iteration (10000 iterations)
[ 36%] Built target bench_benchmarkH3Api
[ 45%] Built target benchmarkGridDiskCells
        -- gridDisk10: 3.901400 microseconds per iteration (10000 iterations)
        -- gridDisk20: 10.393400 microseconds per iteration (10000 iterations)
        -- gridDisk30: 22.128400 microseconds per iteration (10000 iterations)
        -- gridDisk40: 37.856300 microseconds per iteration (10000 iterations)
        -- gridDiskPentagon10: 55.814000 microseconds per iteration (500 iterations)
        -- gridDiskPentagon20: 472.780000 microseconds per iteration (500 iterations)
        -- gridDiskPentagon30: 1610.860000 microseconds per iteration (50 iterations)
        -- gridDiskPentagon40: 4024.100000 microseconds per iteration (10 iterations)
[ 45%] Built target bench_benchmarkGridDiskCells
[ 45%] Built target benchmarkGridPathCells
        -- gridPathCellsNear: 5.172100 microseconds per iteration (10000 iterations)
        -- gridPathCellsFar: 173.823000 microseconds per iteration (1000 iterations)
[ 45%] Built target bench_benchmarkGridPathCells
[ 45%] Built target benchmarkDirectedEdge
        -- directedEdgeToBoundary: 1.260100 microseconds per iteration (10000 iterations)
        -- reverseDirectedEdge: 0.217100 microseconds per iteration (10000 iterations)
[ 45%] Built target bench_benchmarkDirectedEdge
[ 54%] Built target benchmarkGosperIter
        -- hex_plus0: 0.010603 microseconds per iteration (50000000 iterations)
        -- pent_plus0: 0.008550 microseconds per iteration (50000000 iterations)
        -- hex_plus1: 0.103498 microseconds per iteration (500000 iterations)
        -- pent_plus1: 0.090158 microseconds per iteration (500000 iterations)
        -- hex_plus2: 0.368870 microseconds per iteration (100000 iterations)
        -- pent_plus2: 0.344960 microseconds per iteration (100000 iterations)
        -- hex_plus5: 11.754000 microseconds per iteration (1000 iterations)
        -- pent_plus5: 11.500000 microseconds per iteration (1000 iterations)
        -- hex_plus8: 318.720000 microseconds per iteration (100 iterations)
        -- pent_plus8: 310.680000 microseconds per iteration (100 iterations)
        -- hex_plus11: 8574.700000 microseconds per iteration (10 iterations)
        -- pent_plus11: 8374.300000 microseconds per iteration (10 iterations)
[ 54%] Built target bench_benchmarkGosperIter
[ 63%] Built target benchmarkVertex
        -- cellToVertexes: 0.787500 microseconds per iteration (10000 iterations)
        -- cellToVertexesPent: 0.036100 microseconds per iteration (10000 iterations)
        -- cellToVertexesRing: 9.470300 microseconds per iteration (10000 iterations)
        -- cellToVertexesRingPent: 9.475500 microseconds per iteration (10000 iterations)
[ 63%] Built target bench_benchmarkVertex
[ 72%] Built target benchmarkIsValidCell
        -- pentagonChildren_2_8: 146.569000 microseconds per iteration (1000 iterations)
        -- pentagonChildren_8_14: 133.443000 microseconds per iteration (1000 iterations)
        -- pentagonChildren_8_14_null_2: 132.944000 microseconds per iteration (1000 iterations)
        -- pentagonChildren_8_14_null_10: 130.394000 microseconds per iteration (1000 iterations)
        -- pentagonChildren_8_14_null_100: 134.817000 microseconds per iteration (1000 iterations)
[ 72%] Built target bench_benchmarkIsValidCell
[ 81%] Built target benchmarkCellsToPolyAlgos
        -- linked_disk2: 10.811500 microseconds per iteration (10000 iterations)
        -- direct_disk2: 8.677900 microseconds per iteration (10000 iterations)
        -- linked_donut: 4.915700 microseconds per iteration (10000 iterations)
        -- direct_donut: 4.603700 microseconds per iteration (10000 iterations)
        -- linked_nestedDonuts: 19.643600 microseconds per iteration (10000 iterations)
        -- direct_nestedDonuts: 18.380300 microseconds per iteration (10000 iterations)
        -- linked_manyChildren: 2845.100000 microseconds per iteration (10 iterations)
        -- direct_manyChildren: 2796.500000 microseconds per iteration (10 iterations)
        -- linked_colorado: 1513.590000 microseconds per iteration (100 iterations)
        -- direct_colorado: 1481.370000 microseconds per iteration (100 iterations)
[ 81%] Built target bench_benchmarkCellsToPolyAlgos
[ 90%] Built target benchmarkCellToChildren
        -- cellToChildren1: 0.024200 microseconds per iteration (10000 iterations)
        -- cellToChildren2: 0.240100 microseconds per iteration (10000 iterations)
        -- cellToChildren3: 1.590900 microseconds per iteration (10000 iterations)
        -- cellToChildren4: 8.021100 microseconds per iteration (10000 iterations)
        -- cellToChildren5: 49.096800 microseconds per iteration (10000 iterations)
[ 90%] Built target bench_benchmarkCellToChildren
[ 90%] Built target benchmarkPolygonToCells
        -- polygonToCellsSF: 337.670000 microseconds per iteration (500 iterations)
        -- polygonToCellsAlameda: 493.498000 microseconds per iteration (500 iterations)
        -- polygonToCellsSouthernExpansion: 15379.100000 microseconds per iteration (10 iterations)
[ 90%] Built target bench_benchmarkPolygonToCells
[100%] Built target benchmarkPolygonToCellsExperimental
        -- polygonToCellsSF_Center: 231.482000 microseconds per iteration (500 iterations)
        -- polygonToCellsSF_Full: 508.260000 microseconds per iteration (500 iterations)
        -- polygonToCellsSF_Overlapping: 620.610000 microseconds per iteration (500 iterations)
        -- polygonToCellsAlameda_Center: 447.262000 microseconds per iteration (500 iterations)
        -- polygonToCellsAlameda_Full: 1053.464000 microseconds per iteration (500 iterations)
        -- polygonToCellsAlameda_Overlapping: 1421.008000 microseconds per iteration (500 iterations)
        -- polygonToCellsSouthernExpansion_Center: 8758.700000 microseconds per iteration (10 iterations)
        -- polygonToCellsSouthernExpansion_Full: 24479.600000 microseconds per iteration (10 iterations)
        -- polygonToCellsSouthernExpansion_Overlapping: 39002.600000 microseconds per iteration (10 iterations)
[100%] Built target bench_benchmarkPolygonToCellsExperimental
[100%] Built target benchmarkArea
res: 0, diff: 3.552714e-15
res: 1, diff: 5.329071e-15
res: 2, diff: 5.329071e-15
res: 3, diff: 1.776357e-15
        -- allCellsAtRes_print: 27949.000000 microseconds per iteration (1 iterations)
        -- allCellsAtRes_noprint: 19722.000000 microseconds per iteration (10 iterations)
[100%] Built target bench_benchmarkArea
[100%] Built target benchmarks

Grid disk over pentagons seems to be significantly faster (order of magnitude) with this approach. This avoids the possibility of stack overflow by allocating the memory up front instead. Note that h3o's implementation uses a factor of 2.5 instead of 2 for the hash set size. (Cf. https://github.com/HydroniumLabs/h3o/blob/master/src/grid/iterator.rs#L58)

coveralls · 2026-08-09T23:37:07Z

coverage: 99.721% (+0.1%) from 99.588% — isaacbrodsky:griddisk-bfs into uber:master

Port the visited-set half of uber/h3#1217 to the pure-Go safe grid disk traversal. The C PR replaces a recursive fallback with a BFS backed by an open-addressing hash set sized at twice the disk; x/h3go was already a queue-based BFS, so the remaining gap was the Go map used to track visited cells. Replace it with a flat slice hash set (zero Cell as the empty marker, linear probing) and use each completed ring as the queue for the next, which drops the separate queue and its growth. Add BenchmarkGridDiskPentagon to paritytest so the pentagon fallback path is measured at k=10..40. benchstat, Go before vs after (Apple M3 Max, count=6): │ old │ new │ │ sec/op │ sec/op vs base │ GridDiskDistancesSafe/impl=go-16 8.905µ ± 6% 3.858µ ± 3% -56.68% (p=0.002 n=6) GridDiskPentagon/k=10/impl=go-16 51.53µ ± 5% 30.36µ ± 1% -41.09% (p=0.002 n=6) GridDiskPentagon/k=20/impl=go-16 223.6µ ± 2% 121.9µ ± 3% -45.47% (p=0.002 n=6) GridDiskPentagon/k=30/impl=go-16 501.3µ ± 2% 273.4µ ± 4% -45.46% (p=0.002 n=6) GridDiskPentagon/k=40/impl=go-16 939.9µ ± 2% 496.2µ ± 2% -47.20% (p=0.002 n=6) geomean 137.0µ 72.02µ -47.45% allocs/op: GridDiskDistancesSafe 20 -> 4; GridDiskPentagon 30/47/65/91 -> 5. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Port the visited-set half of uber/h3#1217 to the pure-Go safe grid disk traversal. The C PR replaces a recursive fallback with a BFS backed by an open-addressing hash set sized at twice the disk; x/h3go was already a queue-based BFS, so the remaining gap was the Go map used to track visited cells. Replace it with a flat slice hash set (zero Cell as the empty marker, linear probing) and use each completed ring as the queue for the next, which drops the separate queue and its growth. Add BenchmarkGridDiskPentagon to paritytest so the pentagon fallback path is measured at k=10..40. benchstat, Go before vs after (Apple M3 Max, count=6): │ old │ new │ │ sec/op │ sec/op vs base │ GridDiskDistancesSafe/impl=go-16 8.905µ ± 6% 3.858µ ± 3% -56.68% (p=0.002 n=6) GridDiskPentagon/k=10/impl=go-16 51.53µ ± 5% 30.36µ ± 1% -41.09% (p=0.002 n=6) GridDiskPentagon/k=20/impl=go-16 223.6µ ± 2% 121.9µ ± 3% -45.47% (p=0.002 n=6) GridDiskPentagon/k=30/impl=go-16 501.3µ ± 2% 273.4µ ± 4% -45.46% (p=0.002 n=6) GridDiskPentagon/k=40/impl=go-16 939.9µ ± 2% 496.2µ ± 2% -47.20% (p=0.002 n=6) geomean 137.0µ 72.02µ -47.45% allocs/op: GridDiskDistancesSafe 20 -> 4; GridDiskPentagon 30/47/65/91 -> 5. Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>

justinhwang · 2026-09-28T19:05:10Z

+        t_assert(actualAllocCalls == 3, "gridRing called alloc 2 times");
+        t_assert(actualFreeCalls == 3, "gridRing called free 2 times");


nit: update comments

Suggested change

t_assert(actualAllocCalls == 3, "gridRing called alloc 2 times");

t_assert(actualFreeCalls == 3, "gridRing called free 2 times");

t_assert(actualAllocCalls == 3, "gridRing called alloc 3 times");

t_assert(actualFreeCalls == 3, "gridRing called free 3 times");

justinhwang · 2026-09-28T19:07:16Z

 static GeoPolygon sfGeoPolygon;

 SUITE(h3Memory) {
    TEST(gridDisk) {


nit: there is no test that gridDiskDistancesSafe or gridDiskDistances with non-NULL distances returns E_MEMORY_ALLOC when the seen calloc fails

justinhwang · 2026-09-28T19:08:38Z

+        H3Index origin = out[front];
+        if (currK == k) {
+            // Does not need to be explored, edge of disk
+            continue;


I think we can skip the rest here

Suggested change

continue;

break;

isaacbrodsky added 5 commits August 5, 2026 12:23

gridDisk BFS

000f439

make distances optional

a16cc3b

times 2

2ee1fe5

simpler hash

86656c6

Merge branch 'master' into griddisk-bfs

0bd98e5

isaacbrodsky added 3 commits August 9, 2026 17:39

fix leak

7f4b1d1

fix build

1a84bb7

NEVER will maxIdx * HASH_SET_CAPACITY_FACTOR exceed SIZE_MAX

3aa8be3

isaacbrodsky marked this pull request as ready for review September 24, 2026 20:14

isaacbrodsky requested review from ajfriend, dfellis, dmitryzv, jongbinjung and nrabinowitz September 24, 2026 20:14

isaacbrodsky added 2 commits September 24, 2026 13:30

format

fdfe735

Merge branch 'master' into griddisk-bfs

7b721ac

justinhwang mentioned this pull request Sep 28, 2026

perf: open-addressing seen set for safe grid disk BFS in x/h3go uber/h3-go#139

Merged

3 tasks done

justinhwang reviewed Sep 28, 2026

View reviewed changes

Navigation Menu

Repository navigation

Reimplement gridDisk internals with BFS - #1217

Reimplement gridDisk internals with BFS#1217
isaacbrodsky wants to merge 10 commits into
uber:masterfrom
isaacbrodsky:griddisk-bfs

isaacbrodsky commented Aug 9, 2026 •

edited

Loading

Uh oh!

coveralls commented Aug 9, 2026 •

edited

Loading

Uh oh!

justinhwang Sep 28, 2026

Uh oh!

justinhwang Sep 28, 2026

Uh oh!

justinhwang Sep 28, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		t_assert(actualAllocCalls == 3, "gridRing called alloc 2 times");
		t_assert(actualFreeCalls == 3, "gridRing called free 2 times");

Conversation

isaacbrodsky commented Aug 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coveralls commented Aug 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

justinhwang Sep 28, 2026

Choose a reason for hiding this comment

Uh oh!

justinhwang Sep 28, 2026

Choose a reason for hiding this comment

Uh oh!

justinhwang Sep 28, 2026

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

isaacbrodsky commented Aug 9, 2026 •

edited

Loading

coveralls commented Aug 9, 2026 •

edited

Loading