Repository navigation
Commit eb7f085
Benchmark every data type separately, with KeyLoad in each
The site showed KeyLoad only under Multi-model with document numbers: every
database ran one shared document session, and its preparation took 40 minutes.
Benchmarks now runs one job per database, data type and node count (45 jobs).
Each job starts the database through the LoadTests AppHost, runs its type's
scenario and uploads results/<slug>-<type>-n<nodes>.json even when it fails;
Website merges the run's files and renders one table per type.
Each of the ten types (documents, key-value, relational, graph, vector, search,
time-series, analytics, queue, events) has one shared module in
KeyLoad.ComparisonHost: the standard dataset, the scenario and checks, and a
small driver interface. Specialist databases run their type and KeyLoad runs
all ten on the same data.
Removed without replacement: the KeyLoad.Comparisons library, the compiled
runtime bundle, identity and resource evidence, authenticated aggregation,
receipts and GitHub metadata capture, all scripts/Features/BenchmarkComparisons,
the workflow Features actions, the Release benchmark image job, the old website
evidence suite and the superseded benchmark docs and ADRs (ADR-150 records the
new design). The LoadTests AppHost no longer references the server, so benchmark
builds compile only KeyLoad.Client.
KeyLoad gaps the drivers report as real failures: no range access path for
secondary or key ranges without full-scan consent, no SQL UPDATE/DELETE and at
most 256 SQL parameters, no analytics filter, and no AND or indexed lookup in
text search.
Verified: Debug build of KeyLoad.ComparisonHost and KeyLoad.LoadTests.AppHost
with warnings as errors (against the last built KeyLoad.Client, because the
runtime session's uncommitted SDK changes do not build yet); dotnet format and
repository governance; actionlint; local smoke runs with small datasets of Redis
key-value and KeyLoad documents, key-value, relational, graph, vector, search,
queue and events from the released image; WebsiteTests (20) in the site session.
Not verified: full solution build, time-series and analytics locally, and the
first Linux Benchmarks run.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>1 parent 2a8bfef commit eb7f085
1,065 files changed
Lines changed: 9809 additions & 59161 deletions
File tree
- .github/workflows
- Features/BenchmarkComparisons
- BuildIsolatedSite
- DeploySite
- QualifySite
- SessionSetup
- SessionTeardown
- analyzers/KeyLoad.Analyzers
- Analysis/ImplementationTerms
- Contracts
- benchmarks
- KeyLoad.BenchmarkScenarios
- KeyLoad.ComparisonHost
- Analytics
- Cases
- Dataset
- Drivers
- Scenario
- Configuration
- Deployment
- Configuration
- Connections
- Hosting
- Documents
- Cases
- Dataset
- Drivers
- Scenario
- Events
- Cases
- Dataset
- Drivers
- Scenario
- Graph
- Cases
- Dataset
- Drivers
- Scenario
- Hosting
- Identity
- KeyValue
- Cases
- Dataset
- Drivers
- Scenario
- Measurement
- Execution
- Results
- Orchestration
- Assertions
- Cases
- Configuration
- Contracts
- Execution
- Fixtures
- Lifecycle
- Processes
- Reporting
- Scenarios
- Queues
- Cases
- Dataset
- Drivers
- Scenario
- Relational
- Cases
- Dataset
- Drivers
- Scenario
- Reporting
- Search
- Cases
- Dataset
- Drivers
- Scenario
- Session
- Cases
- Configuration
- Fixtures
- Lifecycle
- Reporting
- TimeSeries
- Cases
- Dataset
- Drivers
- Scenario
- Vectors
- Cases
- Dataset
- Drivers
- Scenario
- KeyLoad.Comparisons
- Configuration
- Comparison
- Policies
- Contracts
- Corpus
- Generation
- Diagnostics
- Documents
- Adapters
- Configuration
- Contracts
- Corpus
- Execution
- Lifetimes
- Validation
- Execution
- ResourceMeasurement
- Serialization
- Session
- Configuration
- Contracts
- Corpus
- Execution
- BenchmarkSession
- Models
- Validation
- Targets
- ClickHouse/Execution
- HelixDB
- Contracts
- Execution
- Serialization
- Session
- Vector
- Execution
- Serialization
- HttpDocuments
- Contracts
- Execution
- Hosting
- InfluxDB
- Execution
- Serialization
- KeyLoad
- Contracts
- Corpus
- Diagnostics
- Execution
- Session
- Topology
- Kurrent
- Configuration
- Contracts
- Diagnostics
- Execution
- KurrentCleanup
- KurrentStream
- Lifecycle
- Session
- Topology
- Transport
- MongoDB
- Configuration
- Corpus
- Execution
- Queries
- Schema
- Session
- Topology
- Transport
- Neo4j
- Contracts
- Execution
- Queries
- Session
- OpenSearch
- Contracts
- Execution
- Queries
- Schema
- Serialization
- Session
- Topology
- Transport
- Validation
- PostgreSQL
- Execution
- PostgresSchema
- Session
- Topology
- Vector
- Contracts
- Execution
- Hosting
- Qdrant
- Configuration
- Contracts
- Execution
- Topology
- Vector
- Contracts
- Execution
- Requests
- Transport
- Validation
- RabbitMQ
- Configuration
- Execution
- Topology
- Redis
- Contracts
- Corpus
- Diagnostics
- Execution
- Session
- Topology
- Relational
- Contracts
- Execution
- Hosting
- SurrealDB
- Contracts
- Execution
- Serialization
- Session
- Validation
- Vector/Execution
- TimeSeries
- Contracts
- Execution
- Intensive
- Configuration
- Contracts
- Corpus
- Diagnostics
- Execution
- KeyLoad
- Models
- Queries
- ResourceMeasurement
- Serialization
- Targets/Timescale
- Contracts
- Execution
- Hosting
- Queries
- Schema
- Serialization
- Session
- Validation
- Models
- Serialization
- Targets
- KeyLoad
- Timescale
- TimescaleSchema
- Validation
- Vector
- Contracts
- Corpus
- Models
- Serialization
- Validation
- build/KeyLoad.LoadTests.AppHost
- Configuration
- Contracts
- Hardware
- Identity
- Observations
- Processes
- Resources
- Additional
- Postgres
- SessionTopology
- Serialization
- Validation
- docs
- ADR
- Features
- BenchmarkComparisons
- StorageReplication
- design
- implementation
- scripts
- Features/BenchmarkComparisons
- site
- Features/BenchmarkComparisons
- scripts
- tests/KeyLoad.WebsiteTests
- Assertions
- SiteBrowser
- SiteIsolated
- WebsiteBrowser
- WebsiteBuild
- WebsiteEvidence
- Cases
- SiteBrand
- SiteBrowser
- SiteBuilder
- SiteContent
- SiteCoverage
- SiteDatabase
- SiteHeavy
- SiteIsolatedGitHub
- SiteIsolated
- SiteMetadata
- SiteProcess
- SiteProduct
- SiteQualification
- SiteSession
- SiteVector
- SiteVendor
- SiteWebsite
- Configuration
- Contracts
- WebsiteBrowser
- WebsiteBuild
- WebsiteCoverage
- WebsiteEvidence
- Fixtures
- Results
- benchmark-keyload-analytics-n1
- benchmark-keyload-documents-n1
- benchmark-keyload-events-n1
- benchmark-keyload-graph-n1
- benchmark-keyload-graph-n3
- benchmark-keyload-key-value-n1
- benchmark-keyload-key-value-n3
- benchmark-keyload-queue-n1
- benchmark-keyload-relational-n1
- benchmark-keyload-search-n1
- benchmark-keyload-search-n3
- benchmark-keyload-time-series-n1
- benchmark-keyload-vector-n1
- benchmark-mongodb-documents-n1
- benchmark-mongodb-documents-n3
- benchmark-neo4j-graph-n1
- benchmark-postgresql-pgvector-relational-n1
- benchmark-qdrant-vector-n1
- benchmark-rabbitmq-queue-n1
- benchmark-rabbitmq-queue-n3
- benchmark-redis-key-value-n1
- benchmark-surrealdb-graph-n1
- WebsiteBrowser
- WebsiteBuild
- WebsiteCoverage
- WebsiteEvidence
- Observations
- WebsiteCoverage
- WebsiteEvidence
- Processes
- SiteIsolated
- Scenarios
- WebsiteBrowser
- WebsiteBuild
- WebsiteCoverage
- WebsiteEvidence
- Serialization/WebsiteCoverage
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2 | 2 | | |
3 | 3 | | |
4 | 4 | | |
5 | | - | |
| 5 | + | |
6 | 6 | | |
7 | 7 | | |
8 | 8 | | |
| |||
This file was deleted.
This file was deleted.
This file was deleted.
This file was deleted.
This file was deleted.
0 commit comments