Add random-access decryption to Cobblestone#15322
Open
reaperhulk wants to merge 4 commits into
Open
Conversation
Extend the existing Cobblestone128/256 decryptors with a `decrypt_range` method for authenticated random access into a message, without a new class. The C2SP chunked-encryption wire format is already random-access by construction (each 16 KiB chunk is independently AEAD-sealed with the chunk index bound into the nonce), so this adds only decryption capability; the encryptors and wire format are unchanged. `decrypt_range(source, offset, length)` reads only the chunks covering the requested plaintext range. A `source` may be a buffer-protocol object (e.g. an mmap of a file, for local zero-RAM access) or a binary file-like object exposing just `seek` and `read` (e.g. a small adapter issuing ranged requests to remote/object storage) — one method covers both use cases. To never return unauthenticated bytes, the requested range is silently widened to whole chunk boundaries, every covering chunk is decrypted and authenticated, and only then is the requested sub-range sliced out. A range that cannot be fully authenticated (tampering, or truncation within the range) raises InvalidTag. The commitment is verified once, before any plaintext is released. Streaming (`update`/`finalize`) and random-access decryption are mutually exclusive on a single instance. A range read authenticates the bytes it returns but not the message as a whole, so callers must obtain the total length from a trusted source; this caveat is documented. Rust changes: a stateless `ChunkNonces::nonce_for(index)` (shared with the streaming `next`), a `ChunkCipher::decrypt_chunk_at`, and the `decrypt_range` implementation with buffer/file-like reads. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0172rPrpx6h454zpXTGdzez6
The docs spell-check (sphinx-build -b spelling, warnings-as-errors) only knew the capitalized class-name form "Decryptor"; the new prose in CHANGELOG.rst and docs/cobblestone.rst uses "decryptor" as a common noun. Add the lowercase form to the wordlist. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0172rPrpx6h454zpXTGdzez6
The coverage job flagged five uncovered lines in the new random-access code. Address them: - Remove a now-dead bounds check: the `checked_add(...).filter(...)` guard on the requested range already ensures every chunk index is below MAX_CHUNK_COUNT, so the subsequent `last_chunk >= MAX_CHUNK_COUNT` branch was unreachable. - Add tests exercising the remaining edge branches: a file-like source that signals EOF with None, `update_into` after seeking (mixing guard), a source too short to hold the header, and a final wire chunk shorter than the tag. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0172rPrpx6h454zpXTGdzez6
Fixes an E501 (line too long) flagged by ruff in a test added in the previous commit. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0172rPrpx6h454zpXTGdzez6
alex
reviewed
Jul 25, 2026
| >>> plaintext = b"the quick brown fox" * 10000 | ||
| >>> ciphertext = encryptor.update(plaintext) + encryptor.finalize() | ||
| >>> decryptor = Cobblestone128Decryptor(key, context=b"ranged") | ||
| >>> decryptor.decrypt_range(io.BytesIO(ciphertext), 40005, 9) |
Member
There was a problem hiding this comment.
Having this be an instance method along-side regular decrypt seems super fraught, what happens when you interleave calls?
Comment on lines
+199
to
+205
| :param source: The ciphertext, as either a :term:`bytes-like` | ||
| object (for example an :class:`mmap.mmap` of a file, which | ||
| avoids reading it all into memory) or a binary file-like | ||
| object. A file-like ``source`` only needs to implement | ||
| ``seek(offset)`` and ``read(size)``; nothing else is used, so | ||
| an object that fetches byte ranges from remote storage on | ||
| demand works as well as an open file. |
Member
There was a problem hiding this comment.
a) We should figure out if it really makes sense to do both bytes-like AND file-like. Certainly doing both of them in one-API isn't our usual style. I think you can always turn a bytes in a file-like with io.BytesIO (though maybe that doesn't work for arbitrary buffers?)
b) Gah, is there no python API for pread so we can avoid a reliance on seek? Seek is annoying because it blocks any sort of concurrency.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements
decrypt_range()method for Cobblestone128Decryptor and Cobblestone256Decryptor, enabling efficient decryption of arbitrary byte ranges from large messages without processing the entire ciphertext.Summary
This change adds random-access decryption capabilities to the Cobblestone chunked encryption implementation. The new
decrypt_range()method allows decrypting a specific range of plaintext bytes by reading and authenticating only the 16 KiB chunks that cover that range, making it practical to work with large encrypted files without loading them entirely into memory.Key Changes
New
decrypt_range()method: Added to bothCobblestone128DecryptorandCobblestone256Decryptorclasses. Takes a source (buffer or file-like object), offset, and length; returns authenticated plaintext for the requested range.Helper functions and utilities:
read_at(): Reads bytes from either buffer-protocol objects (direct indexing) or file-like objects (via seek/read)mixing_error()andtruncated_error(): Error helpers for invalid usage patterns and truncation detectionChunkCipher::decrypt_chunk_at(): Decrypts a single chunk at an explicit index without advancing the sequential counterChunkNonces::nonce_for(): Stateless nonce derivation for arbitrary chunk indicesState tracking: Added fields to
ChunkedDecryptorto:decrypt_range()callupdate/finalize) or seeking (decrypt_range) has been used to enforce mutual exclusivityMutual exclusivity enforcement: Streaming and random-access decryption cannot be mixed on a single instance; attempting to do so raises a
ValueError.Comprehensive test coverage: Added 20+ test cases covering:
Documentation: Added detailed docstrings, type hints in
.pyifile, and comprehensive user documentation with examples showing both buffer and file-like usage patterns, including a remote range-fetch adapter example.Implementation Details
decrypt_range()call and cached for subsequent calls.seek()andread()for remote or streaming sources.https://claude.ai/code/session_0172rPrpx6h454zpXTGdzez6