cl: align Gloas consensus and APIs with v1.7.0-beta.2

[domiwei]

Summary

Align Caplin's Gloas consensus behavior and Beacon/Builder APIs with consensus-specs v1.7.0-beta.2, including the missing beta.0 prerequisites required to reach the beta.2 target correctly.

The branch was rebased onto current main at 1ca363730a90ef2419d98f0150e1961d1eb27d84 and reduced from 51 changed files to 37. Operational hardening that is not required for beta.2 has moved to two stacked follow-ups.

Main changes

• initialize anchor PTC votes as uncast and use the parent header slot for payload settlement, attestation processing, and rewards
• enforce pre-Gloas boundaries for proposer preferences, PTC duties, and payload attestations
• validate builder version/activity, parent-requested exits, equal block/parent hashes, gossip timing, and the beta.2 voluntary-exit wall-clock rule
• keep bounded parent builder-exit summaries so untrusted bid validation does not perform unbounded disk I/O, with restart/eviction fallback
• serve canonical FULL execution-payload-envelope ranges with bounded work and explicit unavailable-history behavior
• accept beta.2 invalid transition fixtures without post.ssz_snappy and apply per-case fork config overlays
• pin and pass the official beta.2 mainnet Gloas fixture corpus

Scope split

• cl: recover Gloas payload envelopes at chain tip #24279: chain-tip payload-envelope recovery, persisted replay/revalidation, and checkpoint/P2P plumbing
• cl: harden voluntary exit admission lifecycle #24280: voluntary-exit admission, publication coalescing, eviction identity, and finalized pruning

Both follow-ups remain stacked on this branch and need rebasing onto this rewritten head before their next review.

Latest main rebase

• Base: 1ca363730a90ef2419d98f0150e1961d1eb27d84
• Head: 86f9315cdbaf3a00d2847352dbe54a0e2b6bd50a

Validation

• go test ./cl/phase1/network/services ./cl/phase1/forkchoice ./cl/sentinel/handlers ./cl/beacon/handler -count=1
• go test -tags=spectest ./cl/spectest -run '^Test$/^mainnet$/^gloas$' -count=1
• focused race tests for PTC duties and execution-payload-envelope ByRange handling
• make lint (two clean consecutive runs)
• make erigon integration
• adversarial review covered fork-direction symmetry, zero/nil/genesis boundaries, stale or unavailable state, bounded untrusted work, cache refresh after waits, REST/gossip error-classification boundaries, and local-self-build ingress

The previous review's actionable failures and concurrency findings are covered by focused regression tests. GitHub CI passes on head 86f9315cdbaf3a00d2847352dbe54a0e2b6bd50a.

Devnet 11 experiment

The retained experiment used an older head with embedded Caplin and 15 active Lighthouse validator keys. Attestations and payload attestations were observed, but no proposer duty occurred before the public endpoints and discovered peers became unavailable. The beta.2 head has not been redeployed to the stopped devnet.

Residual risk and deferred work

• the cold envelope disk-read path permits only one active read; acquisition waiters remain bounded by gossip validation concurrency, while a permanently wedged storage call can retain the token until the underlying API becomes cancellable
• full mandatory-range execution-envelope archival remains architectural follow-up; pruned, inconsistent, or incompletely scanned history returns ResourceUnavailable
• broader coordinated forkchoice ordering work tracked upstream in ethereum/consensus-specs#5563, ethereum/consensus-specs#5586, and ethereum/consensus-specs#5125 remains outside this delta

[AskAlexSharov]

Three things to delete, one to move.

cl/phase1/network/services/execution_payload_bid_service.go:456-533 — parentBuilderExitRequests, waitForParentBuilderExits, loadParentBuilderExitRequests, plus the parentExitsCalls map, parentBuilderExitsCall and the parentExitsWork channel hand-roll golang.org/x/sync/singleflight — which this PR already uses, in cl/beacon/handler/handler.go. Group.DoChan keyed by the parent root does the dedup, the wait and the map cleanup; the retry window stays in the LRU. parentBuilderExitsMaxInFlight = 1 means the semaphore only serializes the disk read, and if that is wanted it is one mutex, not a channel plus two double-checked re-locks.

.../execution_payload_bid_service.go:457-465 — the LRU, the calls map and the work channel are built lazily on the first bid, under a mutex, with panic(err) on the validation path. The constructor at :151 already builds validationStateCache. Build them there and the nil check and the panic both go.

cl/phase1/stages/chain_tip_sync.go:480 — retainAcceptedParentEnvelopes spends two structs, two buffered channels, a WaitGroup, a clear and a refill to run at most four OnExecutionPayload calls and drop the ones that fail. maps.DeleteFunc over the map, calling the acceptor inline, is the same behaviour in about eight lines, and it keeps OnExecutionPayload single-threaded the way the block loop called it before this PR. If the parallelism is load-bearing, errgroup.SetLimit(4) says so in three lines.

cl/pool/operation_pool.go:39,61,99 — recentlySeenEntry.identity, restoreIfMissing and recentlySeenIdentityAt have no production caller; operations_pool_test.go is the only one. The identity that recordRecentlySeen stores is never read outside tests, so the field and the parameter threading it through restoreIfMissingWithIdentity go with them. persistentIdentities is the path that is actually used.

Not a finding, a fragility: validateBidAuthentication reads validationStateEntry.state.GetLatestExecutionPayloadBid().BlockHash before the parentVersion >= GloasVersion guard that decides whether the value is used. It holds today because raw.New always allocates the bid, but ProcessExecutionPayloadBid nil-checks the same getter, so the invariant is not trusted elsewhere. Reading it inside the guard, still under the entry lock, costs one line.

[domiwei]

Addressed the review feedback in 0492526e39:

• eagerly initialize the parent-exit cache/coordinator in NewExecutionPayloadBidService;
• gate GetLatestExecutionPayloadBid on the parent fork version and reject a missing bid safely at/after Gloas, with before/exact fork-boundary tests;
• replace the envelope-retention worker plumbing with a bounded errgroup while preserving independent progress for blocked candidates;
• remove the redundant recently-seen identity wrapper and dead restore/test helpers.

I retained the custom parent-exit coordinator instead of replacing it wholesale with singleflight. It owns the disk read independently of caller cancellation and applies a global one-read admission bound across distinct roots; a plain singleflight conversion would only deduplicate equal keys and could allow unbounded different-root work. The existing cancellation, same-root collapse, failed-read collapse, and distinct-root bound tests continue to cover that contract.

The final delta passes the three affected package suites, focused race tests, make lint, and make erigon integration. Two independent boundary/lifecycle adversarial reviews found no actionable issue. Fresh GitHub CI is running on the new head.

[Copilot]

Copilot review overview

🔵 Needs a closer look

The broad consensus, fork-choice, recovery, and concurrency changes require final human review despite extensive targeted coverage.

Review effort: Balanced
Findings: None

What changed in this PR

Aligns Caplin’s Gloas implementation with consensus-specs v1.7.0-beta.0 and strengthens payload-envelope recovery and voluntary-exit handling.

Changes:

• Updates Gloas transition, fork-choice, proposer preference, bid, and attestation behavior.
• Adds checkpoint API envelope recovery and persisted-envelope revalidation.
• Hardens voluntary-exit retry, deduplication, publication, and pruning logic.

File	Description
test-fixtures.json	Updates the consensus fixture archive to beta.0.
cmd/​utils/​flags.go	Propagates integrated checkpoint URLs.
cmd/​caplin/​main.go	Passes checkpoint URLs to standalone Caplin.
cmd/​caplin/​caplincli/​config.go	Stores standalone checkpoint URLs.
cl/​transition/​machine/​machine.go	Updates the payload-bid processor interface.
cl/​transition/​machine/​block.go	Uses the parent header slot during processing.
cl/​transition/​machine/​block_gloas_test.go	Tests parent-slot propagation.
cl/​transition/​impl/​eth2/​operations.go	Aligns bid and parent-payload transitions.
cl/​transition/​impl/​eth2/​operations_gloas_test.go	Tests parent-header settlement behavior.
cl/​spectest/​consensus_tests/​operations.go	Aligns operation fixture processing.
cl/​spectest/​consensus_tests/​operations_test.go	Tests spectest parent-slot selection.
cl/​pool/​operations_pool.go	Adds persistent voluntary-exit identities.
cl/​pool/​operations_pool_test.go	Tests exit identity retention and pruning.
cl/​pool/​operation_pool.go	Adds synchronized identity and pruning support.
cl/​phase1/​stages/​gloas_payload_test.go	Tests persisted anchor-envelope handling.
cl/​phase1/​stages/​clstages.go	Configures checkpoint API recovery endpoints.
cl/​phase1/​stages/​chain_tip_sync.go	Adds bounded HTTP/P2P envelope recovery.
cl/​phase1/​stages/​chain_tip_sync_test.go	Covers recovery, validation, and concurrency.
cl/​phase1/​network/​services/​voluntary_exit_service.go	Hardens exit validation and retries.
cl/​phase1/​network/​services/​voluntary_exit_service_test.go	Covers timing, eviction, and retry cases.
cl/​phase1/​network/​services/​proposer_preferences_service.go	Enforces Gloas boundary and shuffling rules.
cl/​phase1/​network/​services/​proposer_preferences_service_test.go	Tests fork-boundary and dependent-slot behavior.
cl/​phase1/​network/​services/​payload_attestation_service.go	Uses the shared gossip clock allowance.
cl/​phase1/​network/​services/​payload_attestation_service_test.go	Updates clock-boundary tests.
cl/​phase1/​network/​services/​execution_payload_bid_service.go	Strengthens bid and parent-exit validation.
cl/​phase1/​network/​services/​execution_payload_bid_service_test.go	Tests bid validation, caching, and concurrency.
cl/​phase1/​network/​services/​execution_payload_bid_fork_test.go	Tests the Fulu-to-Gloas boundary.
cl/​phase1/​network/​beacon_downloader.go	Adds validated beacon API URL handling.
cl/​phase1/​network/​beacon_downloader_test.go	Tests URL normalization and rejection.
cl/​phase1/​network/​backward_beacon_downloader.go	Reuses validated API URL construction.
cl/​phase1/​forkchoice/​on_execution_payload.go	Revalidates persisted envelopes lacking status.
cl/​phase1/​forkchoice/​on_execution_payload_test.go	Tests persisted-envelope EL validation.
cl/​phase1/​forkchoice/​forkchoice.go	Initializes anchor PTC votes as uncast.
cl/​phase1/​forkchoice/​forkchoice_test.go	Tests uncast anchor votes.
cl/​clparams/​config.go	Adds checkpoint URLs to Caplin configuration.
cl/​beacon/​handler/​pool.go	Makes exit publication failures observable and retryable.
cl/​beacon/​handler/​pool_test.go	Tests exit API validation and publication retries.
cl/​beacon/​handler/​handler.go	Adds publication deduplication state.
cl/​beacon/​handler/​epbs_test.go	Tests proposer preferences at the fork boundary.
cl/​beacon/​handler/​block_production.go	Uses the header slot for attestation rewards.
cl/​beacon/​handler/​block_production_test.go	Tests Gloas reward parent-slot selection.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[yperbasis]

CI is red

[Copilot]

Copilot review overview

🔵 Needs a closer look

It changes consensus-critical transitions, fork choice, gossip validation, persistence behavior, and network protocol handling across many interacting paths.

Review effort: Balanced
Findings: 1

Open (1)

• Bound semaphore waiters to prevent validation goroutine exhaustion

Resolved since last review (2)

• Align client span validation with MAX_REQUEST_PAYLOADS
• Fix spelling of “couldnt” in error message

[domiwei]

Updated #23845 at 676f5778ae and rebased it onto current main (1ca363730a).

The remaining in-scope review items are addressed:

• voluntary exits that fail activity or tenure only because the imported head is stale now return ErrIgnore instead of causing a peer ban
• PTC duty tests align both the handler config and the state-owned config at the Gloas boundary
• a bounded ByRange scan that cannot determine the first FULL payload now returns ResourceUnavailable; partial responses remain allowed once at least one canonical FULL payload has been found
• the outbound ByRange request keeps the beta.2 slot span while bounding response chunks and wire bytes by MAX_REQUEST_PAYLOADS

Local validation on the rewritten head:

• go test ./cl/phase1/network/services ./cl/phase1/forkchoice ./cl/sentinel/handlers ./cl/beacon/handler -count=1
• go test -tags=spectest ./cl/spectest -run '^Test$/^mainnet$/^gloas$' -count=1
• focused race tests for PTC duties and the ByRange handler
• two clean consecutive make lint runs
• make erigon integration

The PR description now records the new base/head and the remaining deferred risks. Fresh CI is running on the rewritten head.

[Copilot]

Copilot review overview

🟡 Changes recommended

Future REST-submitted exits can still be published despite failing the new wall-clock rule.

Review effort: Balanced
Findings: 1

Open (1)

• Prevent gossiping voluntary exits rejected as too early · New

Resolved since last review (1)

• Bound semaphore waiters to prevent validation goroutine exhaustion

[yperbasis]

Re-reviewed at 676f5778ae. The range-scan and stale-head peer-ban findings are fixed. Requesting changes for the remaining REST regression below.

Validation: all ten affected package suites and focused race tests passed. The focused REST regression test passes on base 1ca363730a and fails on this head.

[yperbasis]

Approving at 86f9315cdb. The open items from both earlier rounds and from the ByRange comment are fixed and covered by tests; CI and local package plus race runs are green.

Residual notes, none blocking:

• REST submissions of a wall-clock-valid exit still get a 400 in the epoch-boundary window before the first block of the epoch is imported (cl/phase1/network/services/voluntary_exit_service.go:156 and :166). Stricter than the pre-PR wall-clock check; a validator client retries, so a nit.
• Tip-inclusive ByRange requests still fail closed while the canonical index lags the live head; historical ranges now work during that window.
• The PR body's "Latest main rebase" section lists head 676f5778ae.
• The cold-path semaphore backpressure point stays deferred, as documented under residual risk.