feat(archive): add historical state archive support - #136
Conversation
…uthority/decisions
…ensus foundation)
…tay in chainbase Java 8)
…sensus, no Manager hooks)
…raps, enable-test requirement)
…(txNum across canonical apply)
…ocessBlock writes outside FINALIZE)
… 17 root domains, checksum)
…sets + deterministic map serialization)
…ministic-proto for map-bearing domains, contract abi-strip)
…ception-isolated holder)
…ct store-specific + service wiring + enabled integration test)
… fan-out from account write)
…it, canonical address||slot||version key)
…ced import emits redundant value-correct record)
…ry impl; commit drains capture buffer)
…ekForPrev getAsOf, persistent)
…ath + close lifecycle, default-off safe)
…rted blocks on eraseBlock)
…tomic single-batch unwind, getFirstTxNum)
…t/getCode/getStorage, three-state, no latest fallback, inclusive-after)
…eaderFactory; getStorage uses the codec)
… cursor; survives restart, mirrors temporal store)
…ly block fail-fast)
…ervice (temporal/ + index/ subdirs, close both)
…geAt/getCode via archive (resolver + adapter, latest/disabled preserved)
Bound startup journal scanning and process-wide trace retention, add Bloom/scan tuning and RocksDB observability, reduce disk capacity sampling, and record the completed Round 7 validation matrix.
Harden unified journal integrity, lifecycle shutdown, resource admission, query isolation, and archive-off execution boundaries. Add fault-oriented regression coverage and record the round 10-24 adversarial review evidence.
|
Skipping CodeAnt AI review — this PR changes more than 100 files, which usually means a migration, codemod, or vendored drop. Line-level review on diffs this large produces duplicate findings on the same rewrite pattern and drowns out anything that actually matters. If you still want a review, comment |
❌ Math Usage Detection ResultsFound forbidden usage of Please review if this usage is intended. Caution Note: You should use |
Upgrade grpc-java to 1.83.0, which brings Netty 4.2.15 and fixes CVE-2026-50560 in the HTTP/2 response header path. Close the remaining CVE-2026-47244 pre-ACK enforcement gap in grpc-netty. grpc-netty constructs its HTTP/2 connection directly, bypassing Netty's builder-side early maxActiveStreams initialization. GrpcNettyMaxConcurrentStreamsLimiter now applies the advertised limit before any client stream is accepted, while RpcService still configures SETTINGS through maxConcurrentCallsPerConnection. Use 100 as the finite default stream limit, including when configuration supplies zero. Keep the pooled Netty allocator as FullNode's migration default while preserving an explicit system-property override for deployments that have validated the Netty 4.2 adaptive allocator. Add netty-codec-protobuf for libp2p's Netty 4.1-era protobuf codec references after the Netty 4.2 module split, and complete the new dependency verification metadata. Exclude its protobuf-javanano dependency: netty-parent pins that to 3.0.0-alpha-7, a 2016 release that was discontinued and never patched. Nothing here uses the nano runtime, and framework's binaryRelease packages runtimeClasspath into the shipped fat jars, so without the exclude that artifact would be distributed with FullNode. Cover pre-ACK stream refusal, the CVE-2026-50560 header behavior, invalid stream limits, protobuf codec availability, secure configuration defaults, and allocator override behavior.
java.lang.Math lets the JIT substitute platform-specific intrinsics for several methods, so results are not guaranteed to be bit-for-bit identical across CPUs and JVMs. In a consensus system that non-determinism can fork the chain, which is why all math is required to go through StrictMathWrapper. Replace the regex scan in .github/workflows/math-check.yml with an Error Prone check that runs during compilation. Resolving symbols on the type-attributed AST removes the false positives the regex had on strings, comments and unrelated classes named Math, and it catches forms the regex could not see: fully qualified calls, static imports of both methods and fields, method references, field selects, and the Math.class literal used as a reflection back door. StrictMath is deliberately left alone. Tests cover the thirteen usage forms the checker claims to reject, including the awkward ones (annotation element values, array initializers, anonymous classes, constant initializers), plus the cases that must stay clean: StrictMath, a user class named Math, unrelated members named max/PI, and Math mentioned only inside a string or comment. One test documents the @SuppressWarnings escape hatch. The tests matter more than usual here: the repository has zero java.lang.Math call sites, so if a matcher ever stops firing the build stays green and the only compile-time guard against consensus-relevant math non-determinism disappears without a signal. Verified load-bearing by disabling the matcher, which fails twelve of the eighteen tests. Also correct the class javadoc, which claimed the wrappers are exempted via @SuppressWarnings. No such annotation exists anywhere in the tree, and none is needed: StrictMathWrapper delegates to StrictMath. Note as well that detection is source-level only; reflection on a computed class name remains out of scope, as it was for the regex scan.
Route archive math through StrictMathWrapper, keep tests compatible with JDK 8 and both bundled RocksDB versions, document every archive reference setting, and restore protobuf plugin verification metadata.
Opening the databases dominates startup, and until now the only trace of it sat at debug level, so on a default deployment the node looks idle while it works through dozens of stores. If one of them is slow or wedged there is nothing in the log to say which. Promote the start and done lines in both LevelDbDataSourceImpl.initDB and RocksDbDataSourceImpl.initDB from debug to info, and give RocksDB the same wording as LevelDB so the two backends read alike. Skip the checkpoint databases. They are reopened during normal operation rather than only at startup, so logging them at info would bury the store sequence this change is meant to expose. LevelDbDataSourceImpl already applies that exclusion to its "DB open success" line; this extends the same rule to the surrounding init logs.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4655abb5ac
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if (isNotification(rootNode)) { | ||
| output.discard(); | ||
| writeEmptyResponse(resp); |
There was a problem hiding this comment.
Do not classify every object without an id as a notification. For malformed requests such as {} or {"jsonrpc":"2.0","method":1}, rpcServer.handleRequest can serialize an Invalid Request response with a null ID, but this branch discards it solely because the original object lacks id. This regresses both single requests and batch elements from returning the required error to returning no response; only structurally valid notification requests should suppress output.
Useful? React with 👍 / 👎.
Compile production and test classes with Java 8 bytecode compatibility, update CI to JDK 17, and carry the RocksDB partitioned-filter support needed by the updated build.
Resolve latest-only block selectors before validating call arguments so unsupported tags and missing block objects retain their established JSON-RPC errors.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b18587edc1
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if (project.hasProperty('testJavaVersion')) { | ||
| tasks.withType(Test).configureEach { | ||
| javaLauncher.set(javaToolchains.launcherFor { | ||
| languageVersion = JavaLanguageVersion.of( | ||
| Integer.parseInt(project.property('testJavaVersion').toString())) | ||
| }) | ||
| } |
There was a problem hiding this comment.
Exclude the errorprone project from this JDK 8 launcher override. In the checked docker-build-rockylinux and docker-build-debian11 jobs, clean build passes -PtestJavaVersion=8, so this block also launches :errorprone:test on JDK 8; however, errorprone/build.gradle enables those tests because Gradle itself runs on JDK 17 and supplies Java 9+ --add-exports/--add-opens options. JDK 8 cannot start that test JVM, causing both required x86 builds to fail before validating the application tests.
Useful? React with 👍 / 👎.
Remove the JDK 8 test launcher and Error Prone fallback, require JDK 17 for Gradle builds, and declare JaCoCo class-task dependencies required by Gradle 9.1.
There was a problem hiding this comment.
💡 Codex Review
Preserve the pre-fork x86 Math implementation
When allowStrictMath and the hardened exchange calculation are disabled, ExchangeTransactionActuator deliberately routes consensus exchange pricing through Maths.pow(..., false), but moving the ARM wrapper into the common source set makes that path call StrictMath.pow on x86 instead of the previous Math.pow. These functions can differ by one ULP and change the integer payout—for example, balances 206236204630717/954057449968570 and quantity 120133495269460 produce 351179218498520 with Math but 351179218498519 with StrictMath—so upgraded and older x86 nodes can compute different state on chains where the proposal remains inactive. Keep the x86 implementation until the existing consensus flag selects strict math.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
1 participant
Summary
Supported Historical APIs
eth_getBalanceeth_getCodeeth_getStorageAteth_callValidation
Archive/Historical regression suites, private-chain E2E, restart/offline database checks, and SIGKILL recovery passed. The feature remains disabled by default pending mainnet scale and soak validation.