Repository navigation
Benchmarks: synthetic v2 harness plus Dapper.AOT variant - #140
Merged
Merged
Conversation
…b, V1, V2 to the benchmark project A deterministic 20-column provider of any width and row count, with an optional cost per row and per NextResult and extra empty result sets, and a recorder that times the reader from ExecuteReader to its close and the NextResult drain apart. The four variants are checked for equal final objects by --verify; --selftest-open-time and --selftest-alloc check the timing and that the typed getters allocate nothing. No benchmark is run. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… the synthetic suite Variants over the synthetic provider: V3 (Map then Select, inline), V3s (streamed), V4a and V4b (reused and pooled intermediate row), V5 (Select after the close), V6a, V6c and V6d (massaging inside the map delegate, inline, after the close), V7 (Dapper for the shapes it supports) and V8 (scalars). A consumer-pace parameter busy-waits per item for the variants that run caller code while the reader is open. V0 now drains later result sets like the library's default, so a NextResult cost is charged to every variant alike. --verify runs every variant, and every benchmark method through the benchmark class's own Setup, against an oracle; --selftest-pace shows reader-open time growing with pace only for the inline variants. The project targets net11.0 and net10.0. No benchmark was run. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ns that BenchmarkDotNet's Mean uses The columns were the median of the last 1024 readers, about 6% of the final iteration, so a slow last iteration showed a reader-open time above Mean. The recorder now sums the readers that close inside each WorkloadActual iteration (the engine's own iteration events, in process, nothing allocated per reader), and the column takes the mean per operation over the iterations BenchmarkDotNet's Mean is made of: the report's result runs are matched to their actual iterations, outliers it removed are left out, and a mismatch prints the reason instead of a number. check-readeropen.ps1 fails a report row whose ReaderOpen exceeds Mean by more than 5%. --selftest-open-time gains checks for the stage filter and the matching. No benchmark is run. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…o and the one-off overload timing to the synthetic suite V9c (Db and Syn), V9d, V9d-M and V9e hand variants over the synthetic provider, V6b-F, V6b-E and V6b-K (the constructor path), V9a on/off (rented column buffers, built after the close), V9b (struct rows), V1-fresh (a new RowMap per operation), a DateTimeOffset scenario outside the width ladder and a small class timing the one-off Map overloads against a declared map. --verify checks every new variant against V1, the table or the massaged oracle; the open-time self-test covers V9a's reader-open window. Nothing is benchmarked here. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…synthetic suite The synthetic provider can now model a buffer boundary (BufferRows, RefillCostUs): the sync Read blocks for the refill, ReadAsync returns a truly incomplete task finished from a timer thread, its continuation queued to the thread pool. ThreadPoolPressure keeps K busy items on the pool for the duration of a benchmark case. SyntheticBenchmarks takes the three settings as parameters (ODX_SYN_BUFFER_ROWS, ODX_SYN_REFILL_US, ODX_SYN_PRESSURE). New check modes: --selftest-refill, --selftest-in6, --calibrate-spin. New console mode --in6: the three read paths on SqlClient (LocalDB) and on a SQLite file database, with and without pressure, plus the per-row and per-NextResult calibration. Nothing is run by this commit; all defaults are off. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…scratch folder The probe no longer creates its tables in tempdb. It connects to master, creates a database with a unique name whose data and log files live under the --scratch folder (128 MB data and 64 MB log, 64 MB growth), uses ordinary tables in it, and in a finally sets it to SINGLE_USER, drops it and checks that both files are gone. If CREATE DATABASE fails it prints a STOP line and exits non-zero without falling back to tempdb or the default location; the SQLite part still runs. --scratch is now required; --help and --dry-run show the database and file paths. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…nity and priority --sql-tcp <host,port> with --sql-password-file runs the SQL Server part against a server over TCP as sa (TrustServerCertificate, default packet size). The password is read from the file at run time; it and the connection string are never printed, and every printed exception message is scrubbed. In this mode the dedicated database is created without a FILENAME clause (the server's default location), dropped in a finally, and the files are counted in sys.master_files (0 expected). The log shows the server version and net_transport, protocol_type and net_packet_size of the probe's own connection, and labels the lines SqlServerTcp. --affinity <hexmask> and --priority set the probe's own process affinity and priority at start; the processor count printed afterwards (and the auto pressure K) follows the mask. Without --sql-tcp nothing changes. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…e, build and verify only) RealProviderBenchmarks runs V1, V2, V3, V7 (Dapper) and V9c (typed getters on the concrete reader) over SQL Server LocalDB and a SQLite file database, at width 5 and 20 and 1, 100 and 10,000 rows, with the synthetic catalog's data. The LocalDB tables live in a dedicated database whose files are in a scratch folder (no tempdb, no fallback); the SQLite file is deleted after the case. ReaderOpen is taken by the IN6 timing wrapper (V9c: its own stopwatch on the real reader) and summarized as in the synthetic suite. --verify-rp proves equal final objects per provider against an oracle built from the seed data, and reports loudly when LocalDB is absent. All variants are sync: the stage ranks the mapping paths and does not judge sync against async. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ReadAsync wait time Before S1 the SQL Server part prints the round trip time of SELECT 1 on its own connection (50 warm-up, 200 timed). New options: --scenarios S1,S2,S3 chooses the SQL Server scenarios, --ops-s3 N overrides the measured operations of S3 only; the effective values are in the settings line. The timing wrapper now measures the time spent in incomplete ReadAsync calls (sum per operation, longest single wait) and each cell line shows wait ms sum X max Y, or n/a for a path that never calls ReadAsync. A completed read pays only the completed-flag test. --selftest-in6 checks the wait sum against boundaries x refill on the synthetic provider. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
For every incomplete ReadAsync the timing wrapper notes the row index (the rows already read in that operation). Each async cell line now ends with wait rows first50 [...] hist10 [...] over N ops: the 50 smallest distinct indices with repeat counts, and a 10-bucket histogram over the scenario's rows, pooled over the measured operations. Recording uses preallocated arrays and allocates nothing; a completed read adds a flag test and a row count. The sync path and async execute + Read print n/a. --selftest-in6 checks that a refill every 100 rows over 1000 rows gives positions 100 to 900 and the expected histogram, and none without a refill. --pressure-s3 both|off (default both): with off, S3 runs only its pressure-off cells. The settings line and the dry-run totals show it. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…egate Adds four synthetic-harness variants that read what the hand-written V2 selector reads, built other ways (no benchmark was run): - F0: the simple tier, a compiled expression tree with one try/catch around the row and a harness-private exception that names the column - F1: a compiled expression tree over IDataRecord - F2: the same tree over the sealed concrete reader - F3: the same logic as a static method emitted into a collectible assembly --verify checks all four against V1 and the table at every width and rows 1 and 100, plus a planted bad cell that must be reported with the right column name. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…f|both The timing wrapper now times, with Stopwatch.GetTimestamp and no allocation, the time inside every Read and ReadAsync call (before it returns) and inside every column getter. Each cell line ends with inside ReadAsync ms sum/max, inside Read ms sum/max and getters ms sum (sums are the median over the measured operations, max the longest single call), so a run can tell whether a wait sits in ReadAsync, in Read or in the getters. --selftest-in6 checks it on the synthetic provider: a 500 us refill every 100 rows shows about boundaries x refill inside Read and a small inside-ReadAsync sum, and a getter-heavy fake shows its getter sum. --pressure now also takes off or both (default both): off runs every scenario with pressure off only; auto or an integer still sets K. --pressure-s3 off still applies to S3. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…aReader) Adds two more synthetic-harness variants next to F0 to F3 (no benchmark was run): - F2u: the F2 tree over a copy of the synthetic reader class without `sealed`, with a copy of the connection and command that hands it out - F4: the F1 tree with the reader parameter typed as DbDataReader --verify checks both against V1 and the table at every width and rows 1 and 100, and that the copied reader class really is not sealed. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
P4 reads with ExecuteReaderAsync, ReadAsync and await GetFieldValueAsync<T> for every cell (IsDBNullAsync only for the columns that can be NULL); P5 is the same with CommandBehavior.SequentialAccess. They are hand-written in the probe and build the same row objects as the ODX paths; before a scenario is measured the probe compares the first 5 rows read by P4 and P5 with the sync path and stops if they differ. The timing wrapper counts the async getter calls, how many were incomplete, the wait from return to completion (sum, max) and where they happen (row index: first 50 and a 10-bucket histogram); the cell line shows them as async getters M incomplete of K calls, wait ms sum X max Y getter wait rows .... --paths selects the read paths (default the existing three). The synthetic provider gets SyntheticOptions.BoundaryInGetter (default off): the refill is paid by the first async column read of the boundary row. --selftest-in6 checks that P4 and P5 read the same values and that the incomplete getters equal the number of boundaries. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
P6 uses the provider's own connection and command (no ODX, no timing wrapper): ExecuteReaderAsync, await ReadAsync per row, then sync typed getters, building the same row objects. The probe loop times it inline with Stopwatch: the time awaiting every ReadAsync (sum and longest), the time in the getters, the count of ReadAsync calls that were not complete when checked, and the reader-open time. P6 is selectable with --paths and has the same equality pre-check and pressure handling as the other paths. --selftest-in6 checks it on the synthetic provider. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…hod, FastExpressionCompiler tree) F3r_Sync: F3's emitted static method in a non-collectible (Run) assembly. F1f_Sync: F1's tree compiled with FastExpressionCompiler.CompileFast (6.0.0), failing loudly in setup if it cannot compile. Both verified against V1 and the table; verify also checks the collectibility of the F3 and F3r assemblies. GlobalSetup prints the tiering environment variables once per case. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Adds --s4 <scenario>:<path> (one cell per process, one S4RESULT line and one S4VERIFY line), --s4-setup / --s4-teardown (database and the four tables made server-side with REPLICATE), --s4-rtt and --selftest-s4 (SQLite, the same shapes scaled to KB). Scenarios V1, V2, N1, N2; paths R1 to R5 (raw ADO.NET: Default or SequentialAccess, sync or async reads, R5 with async getters for the large column) and O1 to O3 (ODX selector, generated mapping async and sync). Allocation, peak working set, incomplete async reads and getters with their waits are collected inline. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… --sql-localdb and --s4-scale for smoke tests REPLICATE of a binary value returns varchar, so the V1 and V2 inserts failed with an implicit conversion error. The 1 KB random block now goes through varchar(max) and is converted back with CONVERT; the text tables and the cell warm-up statement use explicit CONVERT as well. Setup checks rows and datalength per table against the repeat counts it used. --s4-scale <divisor> shrinks the values (setup only) and --sql-localdb <instance> connects with integrated security, so the whole flow can be smoke-tested against a local instance in seconds. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…out= The timeout is applied to the command of every cell path: the raw paths set DbCommand.CommandTimeout, the ODX paths call SetTimeout. The warm-up read uses the same value. The self-test records the commands each path creates (a pass-through connection) and checks both the timeout and the result line. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… line With `new`, the first thing the probe does (before any SqlClient type is touched) is set the AppContext switches UseCompatibilityAsyncBehaviour and UseCompatibilityProcessSni to false. Every process prints one S4SWITCHES line with the state read back (unset when not set) and S4RESULT ends with switches=new|default. The self-test checks the helper: default leaves both unset, new sets and reads back both, an unknown mode is rejected. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
SyntheticDapperAot.V1 is the same call as SyntheticDapper.V1 (buffered Query<SynRow>(sql).AsList()), in its own class carrying [DapperAot], so only that one call site is intercepted and V7 stays on plain Dapper. The csproj references Dapper.AOT 1.1.0, allows interceptors in the Dapper.AOT namespace and silences the three advisor codes (DAP219, DAP231, DAP241) that lint the SQL text of the existing benchmark queries. V7a_V1_Sync is a benchmark next to V7_V1_Sync; --verify compares V7a with V1 (the table oracle) and with V2. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Brings the branch up to date with master before the benchmarks PR; only Benchmarks/ differs from master. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…027) - CA2020 (In6Probe): the affinity mask goes through new IntPtr(long), which throws on overflow in a 32-bit process (the checked behaviour of a mask). - CA1068 (SyntheticReader.WaitForGetterRefill): the CancellationToken is now the last parameter; the two callers are updated, nothing else changes. - CA1512 (SyntheticTable.Create): ArgumentOutOfRangeException.ThrowIfNegative. - CA2027 (ChannelStudy): the 30 s Task.Delay gets a cancellation token that is cancelled after the measurement, so no timer is left running. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this adds
The benchmark project gets a synthetic, provider-independent harness and a real-provider cross-check, so mapping paths can be compared on equal terms. Every file is under
Benchmarks/: no change toSource/, tests, docs or build props.SyntheticProvider.csand friends): a 20-column typed catalog (widths 1 to 20, rows 1 to 10,000) with a switchable per-row cost, aNextResultcost, an unsealed twin of the reader, and a buffer-boundary refill with consumer pace.Map(Func<IDataRecord,T>)selector, typed-getter ceilings,Select-style two-step projection, expression-tree and emitted-method forms, plain Dapper, and a Dapper.AOT variant (Dapper.AOT 1.1.0, source-generated interceptors, scoped to its own class so the plain Dapper variant stays plain).--verifyproves every variant produces the same rows as the oracle at every width and row count;--verify-rpdoes the same over SQLite and SQL Server LocalDB; self-tests cover the allocation floor (typed getters at 0 B per row), the open-time recorder, the pace and refill logic.--in6compares sync reads,ReadAsyncand truly async getters, and--s4reads very large values over a simulated link, one cell per process, with the SqlClient opt-in switches selectable.Nothing here ships in a package.
Checks
--verify,--selftest-alloc,--selftest-s4,--selftest-in6and the format check pass; the full test gate and a Visual Studio MSBuild of the solution are green.🤖 Generated with Claude Code