Skip to content

Commit 980c383

Browse files
committed
Summarize each database once in the global benchmark view
Derive common metric rates from total recorded operations and duration, preserve failures and coverage, and restore each data type's original columns. Keep fastest-first sorting and explicit node labels. Verified: website project Debug build with warnings as errors, scoped formatting verification, 26 WebsiteTests, governance and 44 browser sorting checks across all 11 tabs. Canonical solution build and formatting checks are blocked by concurrent uncommitted GraphTraversal/query changes outside this checkpoint.
1 parent 7cda797 commit 980c383

11 files changed

Lines changed: 250 additions & 115 deletions

File tree

‎AGENTS.md‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -339,10 +339,11 @@ Website (`website.yml`) owns only the static landing page. It downloads the
339339
completed run for a site push or manual dispatch, merges them with
340340
`node site/scripts/results.mjs`, builds the static site with
341341
`node site/scripts/build.mjs` and deploys GitHub Pages. The page shows one
342-
comparison table across data types with shared metric columns and KeyLoad in every
343-
type; failed cases read as failed with their error and missing results as no data. Without result files the
344-
merge fails and the current Pages stay. Only GitHub Benchmarks artifacts produce
345-
public figures; local results never do.
342+
global comparison table with each database once and shared metric columns.
343+
Data type tabs retain their original case columns and KeyLoad in every type;
344+
failed cases read as failed with their error and missing results as no data.
345+
Without result files the merge fails and the current Pages stay. Only GitHub
346+
Benchmarks artifacts produce public figures; local results never do.
346347

347348
The landing page has no runtime or qualification dependency on KeyLoad product
348349
code, .NET, Aspire or AppHost. Never restore/build the solution, start containers,

‎docs/implementation/website.md‎

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -20,14 +20,16 @@ inputs, and Website never builds .NET, starts Aspire or runs tests.
2020
when two files describe the same database, type and node count; the workflow
2121
then stops and the current Pages stay.
2222
3. `node site/scripts/build.mjs --results=site-results/results.json --output=_site
23-
--site-revision=<sha>` renders one shared comparison table across data types into static
24-
HTML at build time and publishes the merged file as `data/results.json`.
23+
--site-revision=<sha>` renders a global summary with each database once and the
24+
original per-type comparison tables into static HTML at build time and publishes
25+
the merged file as `data/results.json`.
2526
4. The Pages jobs upload `_site` and deploy it.
2627

2728
The HTML, styles and result modules live in `site/Features/BenchmarkComparisons/`
2829
(`results-contract.mjs`, `results-merge.mjs`, `results-view.mjs`);
29-
`results-controls.mjs` is the only browser script for the results (row filtering by type, fastest-first metric sorting and
30-
table edge shading). The [site protocol](../../site/Features/BenchmarkComparisons/protocol.md)
30+
`results-controls.mjs` is the only browser script for the results (global/type
31+
table selection, fastest-first metric sorting and table edge shading).
32+
The [site protocol](../../site/Features/BenchmarkComparisons/protocol.md)
3133
owns the file shape and cell rendering, and
3234
[BenchmarkComparisons](../Features/BenchmarkComparisons.md) owns the benchmark
3335
contract. `tests/KeyLoad.WebsiteTests` runs the real merge and build scripts

‎site/AGENTS.md‎

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -12,11 +12,13 @@
1212
- `results.mjs` merges the downloaded Benchmarks result files into one sorted
1313
`results.json` (`results-merge.mjs`); `results-contract.mjs` owns the file shape,
1414
types, cases, databases and node counts. `build.mjs` validates the merged file and
15-
renders one comparison table with shared metric columns into static HTML at
16-
build time (`results-view.mjs`), then publishes the merged file unchanged as
17-
`data/results.json`. `results-controls.mjs` is the only browser script for the
18-
results: row filtering by data type, fastest-first metric sorting and table edge
19-
shading. Without JavaScript all rows stay visible in the shared table.
15+
renders a global summary with each database once (`results-summary.mjs`) and
16+
separate tables with each data type's original case columns (`results-view.mjs`).
17+
The summary uses completed operations divided by total measured seconds per
18+
metric and topology; it keeps failures, missing data and case coverage explicit.
19+
The merged file is published unchanged as `data/results.json`.
20+
`results-controls.mjs` selects the global or type table, sorts fastest first and
21+
shades table edges. Without JavaScript all tables stay visible.
2022
- `bootstrap.mjs` loads the hero Three.js scene: a central database, eight connected
2123
data models and the agent above it, with the static artwork as its fallback and
2224
accessible model labels. Bounds are 120,000 triangles, 30 draw calls, 1,000,000

‎site/Features/BenchmarkComparisons/protocol.md‎

Lines changed: 9 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -42,17 +42,19 @@ The builder validates the merged file with the same contract, renders the benchm
4242

4343
## Rendering
4444

45-
`results-view.mjs` renders one comparison table across all data types, with one row per database and data type. Rows have no type groups. The database column stays pinned and identifies the original data type. Shared columns are Load (records/s), Write, Read, Search, Delete, Mixed (ops/s) and Index build (seconds). Each cell retains the name of its original case; a metric containing multiple cases lists them separately. A dash means the type has no case for that metric. Values are never averaged across types or cases.
45+
`results-view.mjs` renders a global summary and one detailed comparison table per data type. The browser selects one table at a time. All types shows each declared database exactly once, including KeyLoad exactly once. Its shared columns are Load (records/s), Write, Read, Search, Delete and Mixed (ops/s). The type tabs use their own original case columns, including Index build in seconds for Search.
4646

47-
Each measured case explicitly labels its 1-node and 3-node values beside their bars. The 3-node speed change says "vs 1 node". Bars scale to the best result of the same original case within its data type. The index build is measured once, so it shows seconds and treats the shortest time as best.
47+
`results-summary.mjs` derives each global rate from the original result files: sum of completed operations divided by sum of measured seconds for the recorded cases in that metric and node count. It does not average rounded rates. Each topology line shows the case count used. Different databases can cover different workloads; the type tables retain the matched scenarios and exact recorded values.
4848

49-
- A failed case reads "Failed" with the first line of its error; the full error stays in the line title and the recorded-values table. A case the job never reached after a job-level error reads "Failed" with that job error.
50-
- A database without a 3-node result shows only its 1-node line and reads "Single node" beside its name.
51-
- A database without any result file, or a case absent from its result, reads "No data" with the reason. Missing values never become zero.
49+
The global metrics map load to Load; update/set/append to Write; point-read/get/pk-select/neighbors/range-read/read to Read; range-query/range-select/traverse/exact-top10/term-query/and-query/aggregate/windows/group-by/filtered-aggregate to Search; delete to Delete; and mixed/cycle to Mixed. A queue cycle is not split into invented independent read or write rates. Index build is a one-time duration and remains in the Search type table.
5250

53-
One collapsed recorded-values table lists every case of every recorded job with data type, status, operations, errors, seconds, operations per second, p50/p95/p99 latency per call, units per call, error, image and run link. The page links the GitHub Actions runs and source commits named in the results and the merged file.
51+
Only recorded jobs contribute to a summary. A recorded job's failed case or job-level error makes that metric read Failed with its error; an absent case within a recorded job reads No data. A metric without a recorded case or a positive measured duration has no number. A dash means the database's declared types have no case for that metric. Missing jobs do not become zero. All original cases and errors remain in the recorded-values tables and the unchanged downloadable results file.
5452

55-
`results-controls.mjs` is the only benchmark browser module. It filters rows by data type in both tables, hides metric columns with no applicable case for the selected type and shades horizontally scrolled table edges. All types share one table. Rows start with the fastest recorded 1-node Load. Selecting another metric sorts fastest first by its first listed original case, using shorter durations for Index build. Failed, missing and inapplicable values follow measured values. The sort basis is visible above the table. Without JavaScript all rows remain visible in their initial speed order.
53+
Every comparison explicitly labels its 1-node and 3-node values. The 3-node speed change says "vs 1 node" and the global summary shows it only when both topologies cover the same original cases. Global bars scale to the fastest summary rate of that metric; type-table bars scale to the best recorded value of the original case. Shorter Index build durations are faster.
54+
55+
A database without a 3-node result shows only its 1-node line and reads Single node. A database with no results reads No data. Each table has a collapsed recorded-values table listing original case status, counts, duration, throughput, latency, batch size, error, image and run. The global recorded-values table contains all types; type tables contain only their own cases.
56+
57+
`results-controls.mjs` selects the summary or original type table and shades horizontally scrolled table edges. Each table starts with its first metric's fastest 1-node value. Selecting a metric always sorts fastest first, using shorter durations for Index build. Failed, missing and inapplicable values follow measured values. The sort basis is visible above each table. Without JavaScript all tables remain visible in their initial speed order.
5658

5759
## Tests
5860

‎site/Features/BenchmarkComparisons/results-controls.mjs‎

Lines changed: 10 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
// One static table; filtering retains its rows and sorting uses the first original case in each metric.
1+
// The global summary and each type own their columns; every selected metric sorts fastest first.
22
function sortMetric(root, head) {
33
const table = head.closest('table');
44
const direction = head.dataset.seconds === 'true' ? 'ascending' : 'descending';
@@ -13,10 +13,11 @@ function sortMetric(root, head) {
1313
});
1414
table.tBodies[0].append(...rows);
1515
const label = head.querySelector('button').firstChild.textContent;
16-
const message = `Sorted by ${label}, fastest first (1 node, first listed case). Failed and missing values follow. Select a column to sort. Swipe or scroll horizontally to see all columns.`;
17-
root.querySelector('.table-scroll-hint').textContent = message;
16+
const message = `Sorted by ${label}, fastest first (1 node). Failed and missing values follow. Select a column to sort. Swipe or scroll horizontally to see all columns.`;
17+
const section = table.closest('section');
18+
section.querySelector('.table-scroll-hint').textContent = message;
1819
const count = rows.filter(row => !row.hidden).length;
19-
root.querySelector('#bench-announcement').textContent = `${root.querySelector('h3').textContent} · ${count} comparison rows. ${message}`;
20+
root.querySelector('#bench-announcement').textContent = `${section.querySelector('h3').textContent} · ${count} comparison rows. ${message}`;
2021
}
2122

2223
// The pinned edge shades once columns pass beneath it; the far edge fades while more columns remain.
@@ -31,22 +32,12 @@ export function shadeScrollRegions(container) {
3132
export function mountResultsControls(root) {
3233
const controls = root.querySelector('.bench-controls');
3334
const results = root.querySelector('#bench-results');
34-
const rows = [...results.querySelectorAll('.ov-row')];
35-
const columns = [...results.querySelectorAll('.ov-head th[data-metric]')];
35+
const sections = [...results.querySelectorAll('.bench-overview')];
3636
const show = () => {
37-
const selected = controls.querySelector('input:checked');
38-
const value = selected.value;
39-
for (const row of results.querySelectorAll('tr[data-type]')) row.hidden = value !== 'all' && row.dataset.type !== value;
40-
for (const head of columns) {
41-
const cells = rows.map(row => row.querySelector(`td[data-metric="${head.dataset.metric}"]`));
42-
head.hidden = !cells.some(cell => !cell.parentElement.hidden && cell.querySelector('.metric-case'));
43-
for (const cell of cells) cell.hidden = head.hidden;
44-
}
45-
const visible = columns.filter(head => !head.hidden);
46-
const table = results.querySelector('.comparison-grid');
47-
table.style.setProperty('--columns', visible.length);
48-
results.querySelector('h3').textContent = value === 'all' ? 'All types' : selected.parentElement.querySelector('span').firstChild.textContent;
49-
sortMetric(root, visible.find(head => head.getAttribute('aria-sort') !== 'none') ?? visible[0]);
37+
const value = controls.querySelector('input:checked').value;
38+
for (const section of sections) section.hidden = section.dataset.type !== value;
39+
const section = sections.find(section => !section.hidden);
40+
sortMetric(root, section.querySelector('th[aria-sort]:not([aria-sort="none"])'));
5041
shadeScrollRegions(results);
5142
};
5243
const sort = event => {
Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
import { NODES, TYPES } from './results-contract.mjs';
2+
3+
export const SUMMARY_COLUMNS = Object.freeze([
4+
{ id: 'load', label: 'Load', kind: 'write', cases: ['load'] },
5+
{ id: 'write', label: 'Write', kind: 'write', cases: ['update', 'set', 'append'] },
6+
{ id: 'read', label: 'Read', kind: 'read', cases: ['point-read', 'get', 'pk-select', 'neighbors', 'range-read', 'read'] },
7+
{ id: 'search', label: 'Search', kind: 'query', cases: ['range-query', 'range-select', 'traverse', 'exact-top10',
8+
'term-query', 'and-query', 'aggregate', 'windows', 'group-by', 'filtered-aggregate'] },
9+
{ id: 'delete', label: 'Delete', kind: 'write', cases: ['delete'] },
10+
{ id: 'mixed', label: 'Mixed', kind: 'mixed', cases: ['mixed', 'cycle'] },
11+
]);
12+
13+
function summarize(results, database, column, nodes) {
14+
const sources = results.filter(result => result.database === database && result.nodes === nodes).flatMap(result => {
15+
const type = TYPES.find(type => type.id === result.type);
16+
return type.cases.filter(item => column.cases.includes(item.id)).map(item => ({
17+
key: `${type.id}|${item.id}`, label: `${type.label} / ${item.label}`, result,
18+
entry: result.cases.find(entry => entry.id === item.id),
19+
}));
20+
});
21+
const base = { nodes, summary: true, caseCount: sources.length, coverage: sources.map(source => source.key).sort().join(',') };
22+
const failed = sources.filter(source => source.entry?.status === 'failed' || (!source.entry && source.result.error !== null));
23+
if (failed.length > 0) {
24+
const error = failed.map(source => `${source.label}: ${source.entry?.error ?? source.result.error}`).join('\n');
25+
return { ...base, status: 'failed', reason: `${failed.length} of ${sources.length} cases failed: ${error.split(/\r?\n/)[0]}`, error };
26+
}
27+
const missing = sources.filter(source => !source.entry);
28+
if (sources.length === 0 || missing.length > 0) return { ...base, status: 'missing',
29+
reason: sources.length === 0 ? 'no recorded cases' : `${missing.length} of ${sources.length} cases not run` };
30+
const operations = sources.reduce((sum, source) => sum + source.entry.operations, 0);
31+
const seconds = sources.reduce((sum, source) => sum + source.entry.seconds, 0);
32+
if (seconds <= 0) return { ...base, status: 'missing', reason: 'no recorded elapsed time' };
33+
return { ...base, status: 'passed', value: operations / seconds, operations, seconds };
34+
}
35+
36+
// Rates describe the recorded workload: completed operations divided by its combined measured duration.
37+
export function summaryModel(results) {
38+
const databases = [...new Set(TYPES.flatMap(type => type.databases))];
39+
const byKey = new Map(databases.flatMap(database => SUMMARY_COLUMNS.flatMap(column => NODES.map(nodes =>
40+
[`${database}|${column.id}|${nodes}`, summarize(results, database, column, nodes)]))));
41+
const line = (database, column, nodes) => byKey.get(`${database}|${column.id}|${nodes}`);
42+
const result = (database, nodes) => results.find(result => result.database === database && result.nodes === nodes);
43+
const best = new Map(SUMMARY_COLUMNS.map(column => [column.id, Math.max(0, ...databases.flatMap(database => NODES.map(nodes =>
44+
line(database, column, nodes))).filter(line => line.status === 'passed').map(line => line.value))]));
45+
const supports = (database, column) => TYPES.some(type => type.databases.includes(database) &&
46+
type.cases.some(item => column.cases.includes(item.id)));
47+
return { summary: true, type: { id: 'all', label: 'All types', databases, cases: SUMMARY_COLUMNS }, results, line, result, best, supports,
48+
nodes: database => NODES.filter(nodes => nodes === 1 || result(database, nodes) !== undefined) };
49+
}

0 commit comments

Comments
 (0)