Skip to content

chore: new benchmark results #73

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
SemyonSinchenko merged 7 commits into from
Oct 1, 2026
Merged

chore: new benchmark results #73

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
1b3714b
benchmarks update
SemyonSinchenko Sep 30, 2026
e4cdec7
chore: formatting
SemyonSinchenko Sep 30, 2026
ae35f7e
chore: formatting
SemyonSinchenko Sep 30, 2026
24476be
chore: update readme
SemyonSinchenko Sep 30, 2026
3b388f6
chore: new data
SemyonSinchenko Sep 30, 2026
ba2fdd8
chore: README
SemyonSinchenko Sep 30, 2026
a5572de
chore: new results
SemyonSinchenko Oct 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
25 changes: 25 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,31 @@

An experimental single node out-of core graph algorithms.

## About

The project aims to re-implement the GraphX computation model ([_Gonzalez, J. E., Xin, R. S., Dave, A., Crankshaw, D., Franklin, M. J., & Stoica, I. (2014). {GraphX}: Graph processing in a distributed dataflow framework. In 11th USENIX symposium on operating systems design and implementation (OSDI 14) (pp. 599-613)._](https://www.usenix.org/system/files/conference/osdi14/osdi14-paper-gonzalez.pdf)) inside the [Apache DataFusion](https://datafusion.apache.org/).

The main idea behind this tool is the question why one should move the data outside the query engine, convert it to Compressed Sparse Rows and then transform it back just to run one of graph algorithms as part of the broader data processing pipeline. And the answer is simple: one doesn't need to do so.

Graph is threated as two relations:

- `edges`
- `vertices`

and algorithms are implemented as relational operations like `SELECT`, `JOIN`, `GROUP BY` and `AGGREGATE` over this two relations.

The "driver" is stateless and does not handle any data bigger a few constants. All the algorithms are written in the way when the peak memory consumption does not depend of the data size but depend only on DataFusion's batch-size and an amount of parallel workers.

## State of the project

Right now there is a CLI that can be used but I cannot guarantee that there won't be any breaking changes in the future. No releases or published artifacts for now. While the libraray public surface is stabilizing, I cannot guarantee as well there won't be any breaking changes.

Feel free to open an issue if you have an idea of what can be chnaged and how the public API should look like.

Python bindings with `pyo3` / `maturin` are planned but out-of-scope right now.

Right now the project is able to run *some* graph algorithms on billion-scale graphs on a sinlge node. See current benchmark results [here](benches/results/README.md).

## Usage

### CLI
Expand Down
24 changes: 12 additions & 12 deletions benches/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,26 +72,26 @@ For every (algorithm, dataset) the runner:

The runner hardcodes LDBC-inspired parameters (no properties/manifest files):

| algorithm | CLI subcommand | parameters |
|---|---|---|
| pagerank | `page-rank` | `--max-iter 10` (damping 0.85) |
| wcc | `wcc` | `--seed 42` |
| kcore | `kcore` | `--max-iter 10` |
| hyperanf | `hyperanf` | `--n-hops 5` |
| sp | `shortest-path` | landmark = 25th percentile vertex id (0.25 * vertices) |
| cdlp | `classical-lp` | `--max-iter 10` (LDBC bidirectional-edge semantic) |
| mis | `mis` | — |
| algorithm | CLI subcommand | parameters |
| --------- | --------------- | ------------------------------------------------------ |
| pagerank | `page-rank` | `--max-iter 10` (damping 0.85) |
| wcc | `wcc` | `--seed 42` |
| kcore | `kcore` | `--max-iter 10` |
| hyperanf | `hyperanf` | `--n-hops 5` |
| sp | `shortest-path` | landmark = 25th percentile vertex id (0.25 * vertices) |
| cdlp | `classical-lp` | `--max-iter 10` (LDBC bidirectional-edge semantic) |
| mis | `mis` | — |

`--undirected` is honoured per algorithm: PageRank is defined only on directed
graphs (ignored), WCC/KCore/MIS and ClassicalLP symmetrize internally
(ignored), and shortest-path + hyperanf get a symmetrized input.

## Monitoring notes

* RSS sampling reads `/proc/<pid>/status` (`VmRSS` per poll, `VmHWM` for the
- RSS sampling reads `/proc/<pid>/status` (`VmRSS` per poll, `VmHWM` for the
peak) — Linux only by design; the JSON simply records no RSS series on other
platforms.
* Disk sampling defaults to `--disk-mode du`: the size of the workdir tree
- Disk sampling defaults to `--disk-mode du`: the size of the workdir tree
itself (checkpoints, spills, output), measured with a pure-python `os.walk`
— exact and cheap (the workdir only holds a handful of parquet files), so it
samples at the same rate as everything else. This is what makes the disk
Expand All @@ -102,7 +102,7 @@ graphs (ignored), WCC/KCore/MIS and ClassicalLP symmetrize internally
free-space accounting (e.g. btrfs) and can read ~0 while the tree holds
hundreds of MB. Deltas are floored at 0 (disk consumption is never
negative), and the RSS/disk plots cap the mean and CI band at 0 as well.
* Point `--checkpoint-dir` at fast local storage (NVMe/SSD); with
- Point `--checkpoint-dir` at fast local storage (NVMe/SSD); with
`--disk-mode du` (the default) the measurement is the workdir tree itself,
so a dedicated mount is no longer required for clean disk numbers.

Expand Down
36 changes: 36 additions & 0 deletions benches/results/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# Setup

## Instance

[`c5d.4xlarge`](https://instances.vantage.sh/aws/ec2/c5d.4xlarge?currency=USD)

| Key | Value |
| --------------------- | ------------------------- |
| vCPUs | 16 |
| Memory (GiB) | 32 |
| Memory per vCPU (GiB) | 2 |
| Physical Processor | Intel Xeon Platinum 8124M |
| Clock Speed (GHz) | 3.4 GHz |
| CPU Architecture | x86_64 |
| NVME Drive | true |
| Disk Space (GiB) | 400 |

## Run arguments

Entry point: `benches/python/main.py`

- `--max-memory 30G`
- `--num-workers 16`

# Results

Wall time, peak RSS and peak disk are medians over 5 runs (see `<algorithm>/<size>/<graph>/benchmark.json`).

| graph | nodes | edges | cdlp | hyperanf | kcore | mis | pagerank (10 iterations) | pic | sp | wcc |
| --------------------- | ----------- | ------------- | ------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| **cit-Patents** (XS) | 3,774,768 | 16,518,947 | <table><tr><td>wall</td><td>12.10 s</td></tr><tr><td>rss</td><td>2.61 GiB</td></tr><tr><td>disk</td><td>0.49 GiB</td></tr></table> | <table><tr><td>wall</td><td>70.71 s</td></tr><tr><td>rss</td><td>21.76 GiB</td></tr><tr><td>disk</td><td>6.56 GiB</td></tr></table> | <table><tr><td>wall</td><td>11.92 s</td></tr><tr><td>rss</td><td>3.63 GiB</td></tr><tr><td>disk</td><td>0.42 GiB</td></tr></table> | <table><tr><td>wall</td><td>9.69 s</td></tr><tr><td>rss</td><td>3.06 GiB</td></tr><tr><td>disk</td><td>0.38 GiB</td></tr></table> | <table><tr><td>wall</td><td>4.05 s</td></tr><tr><td>rss</td><td>1.06 GiB</td></tr><tr><td>disk</td><td>0.33 GiB</td></tr></table> | <table><tr><td>wall</td><td>17.48 s</td></tr><tr><td>rss</td><td>1.83 GiB</td></tr><tr><td>disk</td><td>1.78 GiB</td></tr></table> | <table><tr><td>wall</td><td>0.90 s</td></tr><tr><td>rss</td><td>0.95 GiB</td></tr><tr><td>disk</td><td>0.19 GiB</td></tr></table> | <table><tr><td>wall</td><td>4.71 s</td></tr><tr><td>rss</td><td>1.46 GiB</td></tr><tr><td>disk</td><td>0.23 GiB</td></tr></table> |
| **graph500-24** (M) | 8,870,942 | 260,379,520 | <table><tr><td>wall</td><td>250.23 s</td></tr><tr><td>rss</td><td>29.11 GiB</td></tr><tr><td>disk</td><td>8.24 GiB</td></tr></table> | - | <table><tr><td>wall</td><td>287.82 s</td></tr><tr><td>rss</td><td>25.64 GiB</td></tr><tr><td>disk</td><td>12.76 GiB</td></tr></table> | <table><tr><td>wall</td><td>113.43 s</td></tr><tr><td>rss</td><td>13.50 GiB</td></tr><tr><td>disk</td><td>12.55 GiB</td></tr></table> | <table><tr><td>wall</td><td>24.49 s</td></tr><tr><td>rss</td><td>5.02 GiB</td></tr><tr><td>disk</td><td>2.34 GiB</td></tr></table> | <table><tr><td>wall</td><td>220.52 s</td></tr><tr><td>rss</td><td>12.77 GiB</td></tr><tr><td>disk</td><td>12.35 GiB</td></tr></table> | <table><tr><td>wall</td><td>6.68 s</td></tr><tr><td>rss</td><td>4.97 GiB</td></tr><tr><td>disk</td><td>2.10 GiB</td></tr></table> | <table><tr><td>wall</td><td>33.28 s</td></tr><tr><td>rss</td><td>14.01 GiB</td></tr><tr><td>disk</td><td>6.16 GiB</td></tr></table> |
| **graph500-25** (L) | 17,062,472 | 523,602,831 | - | - | - | <table><tr><td>wall</td><td>242.21 s</td></tr><tr><td>rss</td><td>13.15 GiB</td></tr><tr><td>disk</td><td>28.93 GiB</td></tr></table> | <table><tr><td>wall</td><td>62.32 s</td></tr><tr><td>rss</td><td>12.44 GiB</td></tr><tr><td>disk</td><td>8.59 GiB</td></tr></table> | <table><tr><td>wall</td><td>404.84 s</td></tr><tr><td>rss</td><td>15.18 GiB</td></tr><tr><td>disk</td><td>25.85 GiB</td></tr></table> | <table><tr><td>wall</td><td>26.44 s</td></tr><tr><td>rss</td><td>12.38 GiB</td></tr><tr><td>disk</td><td>8.58 GiB</td></tr></table> | <table><tr><td>wall</td><td>82.46 s</td></tr><tr><td>rss</td><td>18.63 GiB</td></tr><tr><td>disk</td><td>12.20 GiB</td></tr></table> |
| **graph500-26** (XL) | 32,804,978 | 1,051,922,853 | - | - | - | <table><tr><td>wall</td><td>506.16 s</td></tr><tr><td>rss</td><td>13.38 GiB</td></tr><tr><td>disk</td><td>60.63 GiB</td></tr></table> | <table><tr><td>wall</td><td>148.68 s</td></tr><tr><td>rss</td><td>14.26 GiB</td></tr><tr><td>disk</td><td>19.33 GiB</td></tr></table> | <table><tr><td>wall</td><td>883.48 s</td></tr><tr><td>rss</td><td>19.37 GiB</td></tr><tr><td>disk</td><td>59.41 GiB</td></tr></table> | <table><tr><td>wall</td><td>72.86 s</td></tr><tr><td>rss</td><td>13.79 GiB</td></tr><tr><td>disk</td><td>19.86 GiB</td></tr></table> | <table><tr><td>wall</td><td>212.81 s</td></tr><tr><td>rss</td><td>20.52 GiB</td></tr><tr><td>disk</td><td>28.81 GiB</td></tr></table> |
| **graph500-27** (XL) | 65,609,956 | 2,111,642,032 | - | - | - | <table><tr><td>wall</td><td>2790.29 s</td></tr><tr><td>rss</td><td>13.81 GiB</td></tr><tr><td>disk</td><td>124.31 GiB</td></tr></table> | <table><tr><td>wall</td><td>418.51 s</td></tr><tr><td>rss</td><td>15.92 GiB</td></tr><tr><td>disk</td><td>43.82 GiB</td></tr></table> | <table><tr><td>wall</td><td>1896.83 s</td></tr><tr><td>rss</td><td>18.34 GiB</td></tr><tr><td>disk</td><td>120.80 GiB</td></tr></table> | <table><tr><td>wall</td><td>182.62 s</td></tr><tr><td>rss</td><td>17.48 GiB</td></tr><tr><td>disk</td><td>43.67 GiB</td></tr></table> | <table><tr><td>wall</td><td>480.69 s</td></tr><tr><td>rss</td><td>17.28 GiB</td></tr><tr><td>disk</td><td>60.93 GiB</td></tr></table> |
| **graph500-28** (2XL) | 121,242,388 | 4,236,163,958 | - | - | - | - | <table><tr><td>wall</td><td>911.85 s</td></tr><tr><td>rss</td><td>18.31 GiB</td></tr><tr><td>disk</td><td>91.39 GiB</td></tr></table> | - | <table><tr><td>wall</td><td>782.63 s</td></tr><tr><td>rss</td><td>17.59 GiB</td></tr><tr><td>disk</td><td>90.50 GiB</td></tr></table> | <table><tr><td>wall</td><td>1008.91 s</td></tr><tr><td>rss</td><td>20.21 GiB</td></tr><tr><td>disk</td><td>120.34 GiB</td></tr></table> |
Loading
Loading