Skip to content

Commit eb7f085

Browse files
KSemenenkoclaude
andcommitted
Benchmark every data type separately, with KeyLoad in each
The site showed KeyLoad only under Multi-model with document numbers: every database ran one shared document session, and its preparation took 40 minutes. Benchmarks now runs one job per database, data type and node count (45 jobs). Each job starts the database through the LoadTests AppHost, runs its type's scenario and uploads results/<slug>-<type>-n<nodes>.json even when it fails; Website merges the run's files and renders one table per type. Each of the ten types (documents, key-value, relational, graph, vector, search, time-series, analytics, queue, events) has one shared module in KeyLoad.ComparisonHost: the standard dataset, the scenario and checks, and a small driver interface. Specialist databases run their type and KeyLoad runs all ten on the same data. Removed without replacement: the KeyLoad.Comparisons library, the compiled runtime bundle, identity and resource evidence, authenticated aggregation, receipts and GitHub metadata capture, all scripts/Features/BenchmarkComparisons, the workflow Features actions, the Release benchmark image job, the old website evidence suite and the superseded benchmark docs and ADRs (ADR-150 records the new design). The LoadTests AppHost no longer references the server, so benchmark builds compile only KeyLoad.Client. KeyLoad gaps the drivers report as real failures: no range access path for secondary or key ranges without full-scan consent, no SQL UPDATE/DELETE and at most 256 SQL parameters, no analytics filter, and no AND or indexed lookup in text search. Verified: Debug build of KeyLoad.ComparisonHost and KeyLoad.LoadTests.AppHost with warnings as errors (against the last built KeyLoad.Client, because the runtime session's uncommitted SDK changes do not build yet); dotnet format and repository governance; actionlint; local smoke runs with small datasets of Redis key-value and KeyLoad documents, key-value, relational, graph, vector, search, queue and events from the released image; WebsiteTests (20) in the site session. Not verified: full solution build, time-series and analytics locally, and the first Linux Benchmarks run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 parent 2a8bfef commit eb7f085

1,065 files changed

Lines changed: 9809 additions & 59161 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/AGENTS.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
## Purpose, entry points and ownership
44

5-
Owns the four delivery workflows: `build-and-tests.yml`, `benchmarks.yml`, `website.yml` and `release.yml`. Build and Tests owns Release functional qualification; Benchmarks owns isolated database measurements and authenticated aggregation; Website only reads the latest successful benchmark data, generates the static landing page and publishes Pages; Release owns explicitly authorized packaging and publication. Website must never build .NET/AppHost, start Aspire or containers, run product/browser/TUnit/analyzer/coverage suites, depend on product checks, or trigger on product/test-infrastructure changes. Adding such coupling requires specific owner approval. Preserve pinned actions, source identities, original artifacts and least-privilege permissions.
5+
Owns the four delivery workflows: `build-and-tests.yml`, `benchmarks.yml`, `website.yml` and `release.yml`. Build and Tests owns Release functional qualification; Benchmarks runs one job per database, data type and node count and uploads each job's result file; Website only downloads the result files of one Benchmarks run, generates the static landing page and publishes Pages; Release owns explicitly authorized packaging and publication. Website must never build .NET/AppHost, start Aspire or containers, run product/browser/TUnit/analyzer/coverage suites, depend on product checks, or trigger on product/test-infrastructure changes. Adding such coupling requires specific owner approval. Preserve pinned actions, source identities, original artifacts and least-privilege permissions.
66

77
## Boundaries and protected risks
88

‎.github/workflows/Features/BenchmarkComparisons/BuildIsolatedSite/AGENTS.md‎

Lines changed: 0 additions & 29 deletions
This file was deleted.

‎.github/workflows/Features/BenchmarkComparisons/BuildIsolatedSite/action.yml‎

Lines changed: 0 additions & 64 deletions
This file was deleted.

‎.github/workflows/Features/BenchmarkComparisons/DeploySite/AGENTS.md‎

Lines changed: 0 additions & 25 deletions
This file was deleted.

‎.github/workflows/Features/BenchmarkComparisons/DeploySite/action.yml‎

Lines changed: 0 additions & 57 deletions
This file was deleted.

‎.github/workflows/Features/BenchmarkComparisons/QualifySite/AGENTS.md‎

Lines changed: 0 additions & 25 deletions
This file was deleted.

0 commit comments

Comments
 (0)