fix(tests): Address review feedback on test infra changes

[sentry-junior]

Fixes from the retro on #127938, #127939, #127941, #127942 and #127943. Scheduling is unchanged: --dist=loadfile, deterministic order, and xdist 3.8.0 with loadscopereorder = False.

Filtered sharded runs (first commit)

Since #127943, pytest-selected.py plans shards by file before pytest applies -m/-k. Snuba's CI runs sentry's tests with PYTEST_ADDOPTS="-k 'not __In' -m snuba_ci" over 16 shards, so most shards collected 0 tests, exited 5, and cancelled the rest (e.g. getsentry/snuba#8556). With -m/-k in the args or PYTEST_ADDOPTS, every shard now collects everything and keeps its hash share, as before #127943. Sentry's own CI passes neither, so it keeps per-file sharding. Snuba checks out sentry master, so Snuba PRs pick this up on their next run. getsentry/snuba#8556's rerun (attempt 2, started after this merged at 300ebe3) went green, all 16 sentry shards included.

I ran Snuba's collection locally (make test-python-ci args plus the filter, --collect-only, 16 shards). Unsharded, 78 tests are selected:

	Shards with 0 tests	Tests per shard	Union == unsharded
#127943	8 of 16	0–23	yes
this PR	0 of 16	2–8	yes, each test once

Fixes

• Subtests in the JSON report (perf(tests): Use pytest subtests for parametrized tests with heavy setup #127939). Every subtest report was treated as a rerun. A test with 3 passing subtests got 3 reruns entries, and failure comments only showed "contains 1 failed subtest". json_report_reruns.py now keeps one parent setup/call/teardown and puts subtests in a subtests list with their context and longrepr. Real reruns keep their own subtests. Failure comments show each failed subtest's SUBFAILED line and traceback.
• Flush race (perf(tests): Flush transactional test databases with DELETE instead of TRUNCATE #127941). A row committed between the table inventory and the DELETE survived teardown. Cleanup now runs in one transaction that takes TRUNCATE's lock (ACCESS EXCLUSIVE, 1s lock_timeout). If another transaction has touched a table, the lock times out and Django's flush runs, which waits for that transaction the way TRUNCATE always has. That made the pg_stat_activity check redundant, so I removed it.
• Relay readiness (perf(tests): Cut Relay readiness wait in relay_server #127942). The wait no longer overshoots its deadline: the request timeout is min(1s, time left) and no probe starts after the deadline. It catches requests.RequestException instead of Exception. The error includes the last status or exception, and the container is removed on failure.
• Subtest isolation (perf(tests): Use pytest subtests for parametrized tests with heavy setup #127939). test_buffer.py builds a fresh SpansBuffer for each permutation.
• Sharding paths (perf(ci): Collect only each shard's files on full backend runs #127943). Absolute or .. paths also fall back to hash sharding. pytest_shard_plugin raises a usage error if the rootdir isn't the working directory (e.g. --rootdir pointing elsewhere), instead of quietly mismatching paths.

Tests: the JSON report and the Snuba filter run real plugin stacks in a subprocess (the JSON report in-process and with -n 2; the filter checks each test runs exactly once with a Counter). The flush has a writer blocked during cleanup and a writer that's already open at the flush. Relay uses a mocked clock. The JSON report, filter and flush tests fail without their fix.

Declined: an opt-in flag or kill switch for the flush, non-Postgres support, provenance checks for the durations merge, changes to test_client_config.py (no shared state found; reverse order passes all 758 subtests), and docs for running a single subtest.

CI timing

"Run backend test" step across 22 shards, full runs with no test selection (master push and workflow_dispatch):

Run	Sum	Median shard	Max shard
master a636572 (39051886634)	128.5 min	365 s	436 s
this PR before simplification (39059207673)	123.6 min	342 s	424 s
this PR (39061061527)	131.8 min	382 s	441 s

All 22 shards passed in each run. The spread between the three runs (123.6–131.8 min) is runner noise. Files with transactional tests took 34.9 min on master and 36.4 min here (+4%), while all other files took 297.0 and 304.9 min (+3%). Locally, the lock adds about 18 ms per teardown across all three databases (33 ms against 15 ms without it), while Django's flush takes about 1.14 s. So #127941's saving holds.

via @joshuarli.

--

View Junior Session [Sentry]

[github-actions]

🚨 Warning: This pull request contains Frontend and Backend changes!

It's discouraged to make changes to Sentry's Frontend and Backend in a single pull request. The Frontend and Backend are not atomically deployed. If the changes are interdependent of each other, they must be separated into two pull requests and be made forward or backwards compatible, such that the Backend or Frontend can be safely deployed independently.

Have questions? Please ask in the #discuss-dev-infra channel.