Skip to content

Honor explicit autocommit in memory file opens - #2281

Draft
xingzhao-ji wants to merge 3 commits into
fsspec:masterfrom
xingzhao-ji:fix/memory-explicit-autocommit
Draft

xingzhao-ji wants to merge 3 commits into
fsspec:masterfrom
xingzhao-ji:fix/memory-explicit-autocommit

Conversation

@xingzhao-ji

Copy link
Copy Markdown

Summary

MemoryFileSystem._open() currently decides whether to publish writes from self._intrans, ignoring the autocommit value already resolved by AbstractFileSystem.open().

Consequently, fs.open('file', 'wb', autocommit=False) creates the target immediately and leaves its bytes visible after the queued transaction is discarded. Conversely, an explicit autocommit=True inside a transaction does not publish a new file, even though that handle is excluded from the transaction's commit queue.

Use the resolved flag for write/create/append publication and for copying an existing append target. Defaults remain determined by AbstractFileSystem.open(). Add regressions for new and existing files, commit/discard, and both explicit overrides, plus a changelog entry.

Validation

  • All 30 new regression cases fail with unchanged production source.
  • Memory/local/ZIP backend suites: 306 passed, 20 skipped on Linux with both Python 3.12.14 and 3.13.5.
  • Independent review ran the 30 regressions and 32 additional text-mode/store variants successfully.
  • Ruff 0.11.12 check and format checks, and git diff --check, passed.

The dependency-heavy full suite and pinned Ruff environment were not run. The change is limited to write/create/append handling; read/update mode handling is unchanged.

Prepared and tested with OpenAI Codex assistance. No human-review certification is claimed.

@xingzhao-ji
xingzhao-ji force-pushed the fix/memory-explicit-autocommit branch from 84ee38d to 09d1eaf Compare October 13, 2026 20:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant