Skip to content

persistit: do not accept short writes when copying pages to the volume#269

Merged
vharseko merged 1 commit into
OpenIdentityPlatform:masterfrom
vharseko:fix/persistit-volume-short-write
Jul 11, 2026
Merged

persistit: do not accept short writes when copying pages to the volume#269
vharseko merged 1 commit into
OpenIdentityPlatform:masterfrom
vharseko:fix/persistit-volume-short-write

Conversation

@vharseko

@vharseko vharseko commented Jul 11, 2026

Copy link
Copy Markdown
Member

Summary

Fixes #268.

FileChannel#write may report partial progress instead of throwing — the original persistit authors observed this empirically on disk-full (see the comment in JournalManager#flush), and the same reporting is possible for transfers aborted by a concurrent interrupt-driven channel close (CancelIoEx on Windows), the suspected mechanism behind the Windows-only integrity violations in TransactionTest2.transactionsWithInterrupts.

VolumeStorageV2.writePage(ByteBuffer, long) ignored the returned count — a single unchecked _channel.write() call, in contrast to readPage directly above it, which loops until the page is fully read. A short write during copyback therefore silently tore the page in the volume file: intact header, stale tail. The failure chain to the observed silent corruption:

  1. The copier writes the page to the volume on the channel it shares with worker threads; a worker interrupted mid-I/O closes that channel under the copier.
  2. The short count is accepted; cleanupForCopy drops the journal copy, making the torn volume page authoritative.
  3. The pool evicts the page; a later read serves the torn page — Buffer.load() validated nothing beyond a type-byte range check (the page-address and buffer-length checks were disabled Debug asserts, and all checked fields live in the intact header).
  4. Stale record versions in the tail resurrect old values — a silent integrity violation with no exception anywhere, matching the balance mismatches on Windows CI.

Commit/rollback bookkeeping was audited and is exception-safe (abort is marked before any I/O; commit's finally aborts on flush failure), which rules out the transaction paths and leaves torn page I/O as the identified silent-corruption vector.

Changes

  • VolumeStorageV2.writePage(ByteBuffer, long): write until every byte is transferred, mirroring the readPage loop; throw PersistitIOException when no progress is made — safe, because the copier retains the page in the page map and retries the copy on the next cycle.
  • Buffer.load(): turn the buffer-length and page-address Debug asserts into hard InvalidPageStructureException checks, so stale or torn page content fails loudly instead of surfacing later as silently wrong data.
  • ErrorInjectingFileChannel: add a one-shot short-write injection (injectShortWriteOnce) that transfers only the bytes below a given position and returns the partial count without throwing, mimicking an aborted transfer.
  • IOFailureTest#testVolumeShortWriteMustNotTearPage: deterministic end-to-end reproduction — a one-shot short write mid-page during copyBackPages(), then pool invalidation and full read-back verification. Without the fix it fails with CorruptVolumeException: ... page=6 ... is before left edge; with the fix the write loop transparently completes the transfer and all records read back intact.

Testing

  • New test red on the unfixed code, green with the fix.
  • Buffer.load()-sensitive classes (BufferTest, MVCCPruneBufferTest, RecoveryTest, CorruptVolumeTest, WarmupTest, BufferPoolTest, IntegrityCheckTest): all green.
  • Full persistit/core suite: 566 tests, 0 failures, 0 errors, 5 pre-existing skips.

#258 (a Windows skip for transactionsWithInterrupts) was closed in favor of this fix: the test keeps running on Windows, so CI there validates the fix on the affected platform; a residual failure would now surface as a loud InvalidPageStructureException rather than a silent balance mismatch (tracked in #268).

FileChannel#write may report partial progress instead of throwing - the
original authors observed this empirically on disk-full (see the
comment in JournalManager#flush), and the same reporting is possible
for transfers aborted by a concurrent interrupt-driven channel close,
the suspected mechanism behind the Windows-only integrity violations in
TransactionTest2.transactionsWithInterrupts.

VolumeStorageV2.writePage ignored the returned count, so a short write
during copyback silently tore the page in the volume file: intact
header, stale tail. cleanupForCopy then dropped the journal copy, the
pool evicted the page, and a later read served the torn page without
any error, resurrecting stale record versions - matching the silent
balance corruption observed on Windows CI.

- VolumeStorageV2.writePage(ByteBuffer, long): write until every byte
  is transferred, mirroring the read loop in readPage; fail with
  PersistitIOException when no progress is made - safe because the
  copier retains the page in the page map and retries.
- Buffer.load(): turn the buffer-length and page-address Debug asserts
  into hard InvalidPageStructureException checks so stale or torn page
  content fails loudly instead of surfacing as silently wrong data.
- ErrorInjectingFileChannel: add a one-shot short-write injection.
- IOFailureTest#testVolumeShortWriteMustNotTearPage: deterministic
  reproduction; without the fix it fails with CorruptVolumeException
  "page=6 ... is before left edge".

Fixes OpenIdentityPlatform#268
@vharseko vharseko merged commit 04b02d1 into OpenIdentityPlatform:master Jul 11, 2026
14 checks passed
@vharseko vharseko deleted the fix/persistit-volume-short-write branch July 11, 2026 08:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug tests Test code changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

persistit: transaction integrity violated under Thread.interrupt() stress on Windows (ClosedByInterruptException mode)

2 participants