backport: assumeutxo M3 — background validation completion and snapshot promotion

[PastaPastaPasta]

Issue being fixed or feature implemented

M1 (#7451) added AssumeUTXO snapshot persistence and M2 (#7456) gave the snapshot and background chainstates independent EvoDB identities, markers, and chain-aware Dash validation. What was still missing is the end of the lifecycle: nothing ever completed background validation, so a snapshot-backed node stayed in the dual-chainstate state (with DKG participation and quorum signing disabled) forever.

This is milestone 3 of the AssumeUTXO series: background validation completion. When the background chainstate reaches the snapshot base block, the node now verifies the background-derived state against the snapshot, disables the background chainstate, and on the next restart promotes the snapshot chainstate (coins directory and EvoDB markers) to the normal single-chainstate layout.

What was done?

Upstream backports (kept 1:1 where practical, Dash adaptations in separate commits):

• assumeutxo: background validation completion bitcoin/bitcoin#25740 — assumeutxo: background validation completion. Adds ChainstateManager::MaybeCompleteSnapshotValidation() (UTXO-set hash comparison against m_assumeutxo_data when the background tip reaches the base block) and ValidatedSnapshotCleanup() (restart-time promotion of chainstate_snapshot over chainstate), with SnapshotCompletionResult reporting and the chainstate_snapshot_INVALID quarantine path.
• validation: Stricter assumeutxo error handling when renaming chainstates bitcoin/bitcoin#27862 — stricter error handling when renaming chainstates (abort instead of continuing after a failed rename).
• test: make assumeUTXO test capture the expected fatal error bitcoin/bitcoin#28050 — test: capture the expected fatal error during assumeutxo completion failure.
• Rework validation logic for assumeutxo bitcoin/bitcoin#27746 — rework validation logic for assumeutxo: Chainstate::m_disabled replaces ad-hoc usability checks, CompleteChainstateInitialization() split out of LoadChainstate() so chainstates can be reinitialized after cleanup, LoadExternalBlockFile moved to ChainstateManager, per-blockfile undo tracking without the active-chain reference, and BLOCK_ASSUMED_VALID documentation/semantics updates.

Dash-specific completion path:

• EvoDB marker promotion. CEvoDB::PromoteSnapshotMarkers() atomically (single synced batch) moves the SNAPSHOT best-block marker to the legacy NORMAL key and removes all dual-chainstate metadata; DiscardSnapshotMarkers() does the same for a rejected snapshot while preserving NORMAL state. Both reset the transaction-less default identity to NORMAL, closing the TODO(assumeutxo) markers left in M2.
• Base-state comparison. Snapshot activation records a canonical hash of the deterministic masternode list at the base block (EVODB_SNAPSHOT_MNLIST_HASH); the background chainstate independently records the list hash it derives when it connects the base block (EVODB_BACKGROUND_MNLIST_HASH). Completion compares them (in addition to the upstream UTXO-set hash) and fails with SnapshotCompletionResult::EVO_STATE_MISMATCH on divergence. This is the first installment of the holistic base-state comparison M2 deferred; extending it to the CbTx merkleRootMNList/merkleRootQuorums and credit-pool commitments is called out as a TODO for the loadtxoutset milestone, where the snapshot payload gains Dash state. Until then this comparison is a corruption tripwire, not an independent check: in the only case both markers exist (activation with the background tip already at the base) they are written from a single derivation, and a cold-start activation captures neither and skips the comparison. Divergence therefore only signals on-disk damage to the marker pair; the independent comparison arrives when the snapshot payload carries the base MN list.
• Crash recovery. ValidatedSnapshotCleanup() performs two directory renames plus a marker promotion, each individually durable. RecoverSnapshotCleanup() (run at startup before chainstate detection) classifies every interruption point — first rename done, both renames done with markers pending, promotion durable but deletion pending, invalid-snapshot rename done with marker discard pending — and either rolls back, finishes the promotion, or fails with a precise error instead of the generic reindex advice.
• Lifecycle correctness fixes discovered while wiring the above: EraseSnapshotMarkers() (the abandoned-activation rollback from M2) now also erases the new MN-list-hash markers; snapshot activation moves the mempool to the snapshot chainstate and restart activation clears it from the background chainstate (the assumeutxo (2) bitcoin/bitcoin#27596 shape), so background block connects can no longer call removeForBlock/removeExpiredAssetUnlock against mempool state built on the snapshot tip; the invalid-snapshot revert hands the mempool back.

Review follow-ups from the M2 merge applied here:

• The peer-penalty exemption for unavailable history no longer depends on three files repeating one literal string: the sentinel is a named constant (BLOCK_DATA_UNAVAILABLE_SUFFIX) shared by every producer and the matcher.
• The background MN-list hash is computed only for the snapshot base block (activation captures it directly when the background tip is already at the base). The initial implementation hashed the full deterministic MN list on every block connect, which would have been a measurable IBD regression on every node.
• A comment documents the cross-chainstate duplicate-commitment corner in CQuorumBlockProcessor::ProcessCommitment.

Review follow-ups from the #7553 review round:

• A snapshot base block missing from the on-disk block index is now reported by ChainstateManager::LoadBlockIndex() as a normal startup failure (recoverable via the standard reindex advice), instead of aborting in candidate admission; GetSnapshotBaseBlock() regains upstream's cached SnapshotBase() delegation (bitcoin d4a11ab) that the initial adaptation had dropped. Note the related deliberate deviation: MaybeCompleteSnapshotValidation() converts one upstream hard assert into a SKIPPED return for synthetic in-memory unit fixtures, discriminated by CoinsDB().StoragePath() being empty.
• Every snapshot-lifecycle directory rename/removal now goes through new RenameDurably()/RemoveAllDurably() helpers (fs::rename/fs::remove_all + DirectoryCommit), so the crash-recovery invariant is enforced by the helper rather than by remembering a follow-up call at six sites.
• New tests: a missing-base startup failure, the promote/discard overlap state in RecoverSnapshotCleanup (must land in the same end state as a completed discard), and background-MN-hash coverage in the marker rollback test.

With completion wired, the M2 duty gate resolves end-to-end: IsSnapshotActiveAndUnvalidated() becomes false at completion, so DKG participation and quorum signing re-enable without a restart, and the masternode status clause clears.

How Has This Been Tested?

• Rebased onto develop immediately after the M2 merge (ab65592f85d); every conflict was resolved against M2's final review round (thread-scoped EvoDB transactions, EraseSnapshotMarkers, reindex-time snapshot discard, fallible DetectSnapshotChainstate, BLS scheme establishment). The merged M2 test chainstate_connectblock_bls_scheme is adapted in the Rework validation logic for assumeutxo bitcoin/bitcoin#27746 commit for AcceptBlock moving to ChainstateManager.
• Full clean build (autotools, --enable-debug), then the complete test_dash suite passes ("No errors detected"), including targeted reruns of evo_db_tests, validation_chainstatemanager_tests, validation_chainstate_tests, evo_deterministicmns_tests, evo_mnhf_tests, evo_assetlocks_tests, evo_cbtx_tests, blockmanager_tests, coinstatsindex_tests, and validation_block_tests.
• New coverage: snapshot_marker_promotion_and_discard (promotion/discard idempotency across restarts), the extended abandoned-activation marker rollback test, chainstatemanager_snapshot_completion and _hash_mismatch (upstream-shaped), an EVO_STATE_MISMATCH completion case, four crash-recovery tests that each reproduce a distinct ValidatedSnapshotCleanup interruption point on disk and drive it through LoadVerifyActivateChainstate(), and mempool-ownership assertions at both activation paths.
• An independent review pass traced the highest-risk interactions end to end: the ConnectTip → MaybeCompleteSnapshotValidation EvoDB transaction lifecycle (the scoped committer closes before completion runs, so the single-open-transaction invariant holds), BLS-scheme guard nesting across connect/disconnect, all four mempool handoff transitions, and the recovery state machine. Its two "correct but implicit" findings are addressed in the final commit (at-rest raw reads for the lifecycle markers; a comment documenting the deliberate promote/discard overlap in RecoverSnapshotCleanup).
• lint-circular-dependencies, lint-python, and git diff --check are clean.

Breaking Changes

None released. The dual-chainstate on-disk state introduced in M2 (unreleased) gains two lifecycle marker keys (b_dcs_mn, b_dcs_bg_mn); nodes that never load a snapshot never write any of them. ChainstateLoadStatus::FAILURE_FATAL is a new internal failure class treated like FAILURE_INCOMPATIBLE_DB at init.

Checklist:

• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have added or updated relevant unit/integration/functional/e2e tests
• I have made corresponding changes to the documentation (doc/design/assumeutxo.md updated for the implemented lifecycle; the user-facing AssumeUTXO documentation lands with loadtxoutset)
• I have assigned this pull request to a milestone