Skip to content

Run converted legacy cases through the operator case runner - #4544

Open
evanchooly wants to merge 1 commit into
audits-case-manifestfrom
audits-case-adapter
Open

evanchooly wants to merge 1 commit into
audits-case-manifestfrom
audits-case-adapter

Conversation

@evanchooly

Copy link
Copy Markdown
Member

Stacked on #4543 (step 5, slice 3 of the test generation design).

Problem

The runner from #4542 only ran tests written in its own form, and moving a legacy body into that form (as $round was) doesn't scale to 556 cases. Legacy cases need to join the runner with their bodies untouched, so that they gain the strict comparison and the reference-driver behavior check without regenerating reviewed code. The plan calls this the compatibility adapter.

Change

Core.

  • The adapter: TemplatedTestBase.testPipeline checks whether the case's fixture directory holds a scenario.json. If it does, it hands the test's unchanged lambda to OperatorCaseRunner.
  • Options check: first, the test's ActionTestOptions must say what the scenario says: result order, no removeIds or skip options, no version requirements. A mismatch fails the test, since the scenario now drives the run.
  • Guards: testQuery and testUpdate refuse a converted fixture until they're supported.
  • Lookup and completeness: OperatorCase accepts scenarios in legacy fixture directories. GeneratedCases knows the CONVERTED kind, walks legacy fixture directories for unlisted scenarios, and checks every case's @DisplayName against the scenario's documented title. That title replaces the deleted name file.

Audits.

  • The command: FormMainKt convert <root> <revision> <fixture> replaces a case's legacy files with the scenario written from its accepted, exact override. Only testPipeline cases are accepted, with no options beyond orderMatters.
  • Ledger fields: the migration ledger records kind CONVERTED, the legacy method, which stays, and the legacy body text, checked against the frozen catalog's digest. The result order is re-derived from the body text.
  • Backfill: $round's entry gains its body text from Git history, which closes the item deferred in Add the operator case runner and move $round onto it #4542.
  • Ownership: the manifest lists converted cases, and the owners check counts a legacy binding and its converted case as one owner.
  • Inventory: FixtureMigration.retiredLocks, shared by RepairsMain and its test, keeps a converted case's retired lock classifying it in the inventory.

First converted case. trunc/truncExpressionOperator passes all three checks under both mappers with its body unchanged; its legacy files are gone.

Decisions for review

  • Scope. The adapter covers aggregation cases whose reference is an accepted, exact override. These come with the cutover slice instead, where every case's reference is decided:

    • references for never-locked cases;
    • CURATED_REFERENCE and SOURCE_REMOVED;
    • query and update cases.

    Until then they're refused rather than half-supported.

  • Ordering detection. The result order is detected from a literal .orderMatters(false) in the body. That fails closed: convert cross-checks the inventory's options, and the runner checks the real options at run time.

Verification

  • Core aggregation, query, generated, runner and manifest tests: 656 under Critter, then reflection.
  • Audits suite: 405 under both mappers.
  • The form report lists 136 overrides; two cases have moved into the ledger. The inventory stays at 555 examples, 137 OVERRIDDEN and 98 locked.
  • Two local review rounds. Round 1 found that converting dropped the display-name check. It's restored through the scenario title, with both scenarios regenerated by the tool. Round 2 found nothing new.

A legacy testPipeline case whose fixture directory holds a scenario.json
runs its unchanged lambda through OperatorCaseRunner, after its options
are checked against the scenario. FormMainKt convert replaces a case's
legacy fixture files with the scenario written from its accepted override,
and the migration ledger keeps the legacy method, its hash-checked body
text and the result order derived from it. Scenarios carry the documented
example title, which each case's @DisplayName must equal.
trunc/truncExpressionOperator is the first converted case.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant