Skip to content

Preserve None values in filesystem serialization - #2264

Open
fhgffy wants to merge 3 commits into
fsspec:masterfrom
fhgffy:fix/20261011-file-handling
Open

fhgffy wants to merge 3 commits into
fsspec:masterfrom
fhgffy:fix/20261011-file-handling

Conversation

@fhgffy

@fhgffy fhgffy commented Oct 10, 2026

Copy link
Copy Markdown

Explicitly passing a nullable filesystem option currently breaks both to_dict() and to_json():

from fsspec.implementations.http import HTTPFileSystem

HTTPFileSystem(block_size=None, cache_options=None).to_dict()
# TypeError: Object of type NoneType is not JSON serializable

The recursive pre-serialization pass sends None to the encoder's unsupported-object handler. Preserve it alongside the other JSON primitives so it is represented as null in JSON and remains None in the dictionary form.

The regression tests cover positional arguments, keyword options, nested lists and mappings, exact serialized output, round trips, unchanged constructor state, and a real HTTPFileSystem configuration.

Validation

Linux, CPython 3.12.14, upstream 94afb3f6e99d1b74dffb58811f8a06fb7c27ed27:

  • All 9 new regression cases fail on the original implementation and pass with the fix.
  • python -m pytest fsspec/tests/test_spec.py -q --benchmark-disable: 388 passed, 10 skipped, 1 xfailed.
  • python -m pytest fsspec -q --benchmark-disable --maxfail=8: 2220 passed, 196 skipped, 2 xfailed, 2 failed. The two failures also reproduce on an unmodified checkout of the same upstream commit: test_processes[forkserver] cannot create an AF_UNIX socket in this environment, and test_async_cat_file_ranges cannot resolve the external NOAA S3 hostname.
  • Ruff check, Ruff format check, and git diff --check pass.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant