Conversation
…ined Three places the report layer was losing something it had been handed. A component that produced nothing reached `collect` as `None`. The reason was known — a `GaveUp` carries the author's own account of what stopped it, and a crash carries its exception text — and both were discarded at the boundary because the input type had nowhere to put them. `Abandoned(reason)` is that variant, `GaveUpComponent.reason` is where it lands, and `_abandonment` in the pipeline picks the better of the two sources. This is the one change here with an immediate effect on every run: a give-up in the report now says why. `BuildEnvironment` is the second. A backend that compiles the project under verification can be told not to confine those builds, which makes every verdict it then produces a development result rather than a production one — and the only record of that was stderr on whatever machine ran it. `ConfinedBuilds | UnconfinedBuilds` is a union rather than a flag because the confined case names its mechanism and the other carries nothing, and the field is optional rather than defaulted because absent is a third state: an EVM backend compiles nothing of the project and has no build to have confined, and a report written before schema 3.3 cannot speak to it either. The renderer states only what it was told, and marks the unconfined case in the header. `Formalizer.build_environment()` is the hook that supplies it; every backend on master inherits the `None`. Third, `make_prover_fetcher` was annotated at `GeneratedCVL` while reading nothing but `run_link`, so a backend wanting the same POU roll-up had to reach past it for the private function. It is typed at `ReportableResult` now and `fetch_verdicts` is public. Alongside it, `job_input` handles a job link shape POU's URL parser does not know, by handing over the job id POU also accepts. That half has no caller on master — the shape is the Solana Prover's — but it belongs with the fetcher rather than alone, and the failure it prevents is silent: POU raises, the best-effort fetch swallows it, and every rule of a green run is reported UNKNOWN. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Review pass over c95deaa and e6f9e2e. Comments describe the code as it is; how it got there belongs here, in the commit. Eight comments narrated the edit instead of the code -- the CVLR run that motivated GaveUpComponent.reason, the backend that reached past make_prover_fetcher, the twelve verdicts re-composed on the code the draft gate replaced, and five shorter "it used to be None" asides. The claim each one was making about the code survives; the history is dropped. "stderr on a machine nobody kept is not a record" appeared three times. It is a line, not information, and the sentences around it already carry the point. render.py's build_environment comment restated schema.py's verbatim; trimmed to the template-side fact. Two comments the drafted/did_read rename missed: ui_harness_natspec named validator `did_rough_draft_read` and the read it no longer requires, and test_autoprove_report's module docstring still described a gap as `None`. test_the_fetcher_serves_any_reportable_result asserted only callable(fetcher), so its docstring claimed more than it checked. It now runs the fetcher over a non-CVL Formalized, both the verdict path and the run_link is None one. Four completion-rejection messages still told the agent to read the draft and call read_rough_draft. They fire on `not drafted`, which a write now satisfies, so they were instructing a rejected agent to make exactly the extra round-trip e6f9e2e removed. Every .j2 prompt was updated there; these four were not. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Recording whether a build ran confined was carried here from the Solana work on the assumption that a reader of the report would want the caveat. It does not earn its place: nothing consumes it, no backend on master can set it, and a field that only ever renders absent is a schema commitment bought with nothing. `schema_version` therefore moves one step rather than two: 3.2 is `GaveUpComponent.reason` and nothing else. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
152e254 reached six files this PR otherwise does not change: the four completion-rejection messages still asking for a read_rough_draft, the natspec tape's comment about the validator that replaced, and a docstring trim in the rough-draft tool tests. All of it is correct and none of it is about the report, so a reviewer here would be reading it cold. Restored to master's text. The rough-draft message fix is worth landing where it is the subject. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ericeil
marked this pull request as ready for review
September 16, 2026 23:21
ericeil
added a commit
that referenced
this pull request
Sep 18, 2026
All three are out of draft. S1 (#239) and S4 (#241) are awaiting a first read; S3 (#240) has changes requested, with six of its eight threads outdated by the commits that answered them and two still live as questions. S3's sizes moved most: the ProverOptions rework came out of that review, so the PR is 28 files where the plan recorded 16. S4's moved the other way — the build-environment field was dropped before review, taking report/render.py and the report template out of its file set. Two path lists were stale beyond the counts. S1 reaches composer/foundry/entry.py and spec/source/autoprove_common.py, and S3 reaches the latter too — three lines apart in the same ProverBackend construction, which is a merge to pay rather than a hunk to pick, so the appendix no longer claims every path appears once. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ericeil
added a commit
that referenced
this pull request
Sep 18, 2026
#239 merged as 27f2fe9 and #240 as 568ba02, half an hour apart on 2026-09-17. S3 answered its changes-requested review by absorbing the ProverOptions rework, which is most of why it nearly doubled between opening and merging. That leaves S4 as wave 1's only open PR, and it inherits both: S1's ecosystem parameter sits a few lines from the give-up boundary S4 retypes in pipeline/core.py, and S3's cex_dump envelope is in a test file S4 otherwise rewrites. merge-tree reports no textual conflict, but #241 has not been gated against the new head — the caution about those files is rewritten to say so rather than to describe an ordering that is now settled. The autoprove_common.py collision between S1 and S3 resolved itself in the merge order. The cli.py cwd bug fix is still unlanded and still belongs on its own. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Third in a series splitting long-lived Solana/CVLR work into independently reviewable pieces (after #239 and #240). AutoProver cannot run the Solana Prover yet; this is shared report work that has to be in place before a Solana backend can produce a report anyone can trust.
Two different layers are involved, and they read more easily apart:
composer/spec/source/report_prover.pyis the adapter between those layers, which is the only reason one file appears in both halves below.Getting verdicts back from the Prover
prover_output_utilityextracts a job id from/output/<user>/<job>/…or/job/<job>and recognizes no other shape. The Solana Prover's report link — the onecertoraSolanaProverprints and the CLI records — is/jobStatus/<userId>/<jobHash>?anonymousKey=…, so the extraction raises.The report's verdict fetch is deliberately best-effort: any failure means "no verdicts for this run". Together those two facts make a completed Solana job come back empty. On the development branch, a run whose own
output.jsonreported SUCCESS for all seven rules produced a report in which all seven were UNKNOWN, with no source lines and no durations — a false negative in the deliverable, arrived at silently, from a job that was green.POU's entry points accept a bare job id as well as a URL, so
job_inputextracts the id and hands that over. Every shape POU already parses passes through untouched, which the tests pin — including the localemv-path its offline mode wants, because fixing the Solana link must not break the EVM one.Two things to be clear about: the real fix belongs upstream in POU, and nothing on master reaches this path, since the shape is the Solana Prover's. It is here rather than in a PR of its own because it lives in the same adapter as the type change below.
What the report document records
A component that gave up now says why
When AutoProver stops on a component, it usually knows exactly what stopped it — often naming the prover error and the setting that would have unblocked it. The report recorded only that the component produced nothing: the reason existed on both paths (the author's own account, or an exception's text) and was discarded at the boundary, because the input type for "produced nothing" was
Noneand aNonehas nowhere to put it.Abandoned(reason)is that variant, andGaveUpComponent.reasonis where it lands.This is the one change here that affects AutoProver today, on EVM runs as much as any other, and the reason is often the most useful line in the document.
The verdict hook no longer names one backend
make_prover_fetcherreads nothing but a run link, but was typedVerdictFetcher[GeneratedCVL], so any other backend wanting the same roll-up had to reach past the factory for the module-private_fetchand re-implement therun_link is Nonecheck around it. It isVerdictFetcher[ReportableResult]now, and_fetchbecomes public asfetch_verdicts. The test drives the returned fetcher over a non-CVLFormalized, on both the verdict path and therun_link is Noneone.Review notes
schema_version→3.2, forGaveUpComponent.reason. Earlier revisions of this PR also carried abuild_environmentfield recording whether a backend's builds ran sandboxed; it has been dropped, because nothing consumes it and no backend on master can set it.composer/pipeline/core.pyis also touched by Let a component lose property extraction without losing the run #228; the report files by Generalize findings to support non-Prover backends #185 and Read verdicts from every run that accounts for the published spec #232. Whoever lands second pays a small merge.pytest -m "not expensive"→ 1336 passed;pyright→ 0 errors.🤖 Generated with Claude Code