Repository navigation
Design generated operator tests (step 5) - #4541
Open
evanchooly wants to merge 1 commit into
Open
evanchooly wants to merge 1 commit into
evanchooly wants to merge 1 commit into
Conversation
A descriptor-driven runner checks serialization, behavior and published results separately. Legacy cases join it through a compatibility adapter with their documented sources as references; generation covers examples no test owns. Fixes the identity, scenario, manifest, migrated-fixture, candidate and promotion contracts and the slice order.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #4540. Design only:
design/operator-audits-test-generation.md, the plan for step 5 of the operator audit plan.Problem
Step 5 asks for executable Morphia tests generated from the documented examples. That's typed DSL code, checked separately for serialization, behavior and the published result, without erasing handwritten tests. Nothing for it exists yet.
Measurement. Against a 2025
mongodb/docscheckout there are about 550 documented operator examples. About 521 are already owned by the 556 legacyTemplatedTestBasecases. The ~29 unowned ones are mostly admin stages that need an environment the tests don't have (sessions, search indexes, sharded data distribution), plus the positional update operators, whose fixtures are named differently. Generating tests only for unowned examples would therefore cover almost nothing today.Design
OperatorCaseRunner, built onTestBase, with anOperatorCasedescriptor and onescenario.jsonper case. It checks three things separately:CURATED_REFERENCEand aren't counted as parity.scenario.jsonin the same change, owned by the generated manifest. A migrated-fixture ledger declares each removed file, so the existing audits checks keep guarding fixture bytes.$round).Decisions for review
$round) verbatim into the new form, so the runner is proven before the adapter exists.Review
Three local review rounds; the progress journal lists what each changed.