Repository navigation
Add Python 3.14 support (fix forkserver CI hang) - #1152
Merged
rkingsbury merged 2 commits intoOct 27, 2026
Merged
Conversation
Re-enables Python 3.14 support, which was previously added alongside the 3.9/3.10 drop (8c1e3c6) and then removed from CI in 942efbc pending diagnosis of a CI hang. The 3.11-3.14 source compatibility fixes made on that branch (asyncio event-loop, generator pattern, datetime.utcnow) remain in place on main. - pyproject.toml: add the Python 3.14 classifier (requires-python is already >=3.11, so no bound change needed) - testing.yml: add 3.14 to both the `test` and `test-min-deps` matrices - upgrade-dependencies.yml: add 3.14 so pinned requirements regenerate - requirements: restore the py3.14 lock files (ubuntu ones are real cp314-compatible pins, e.g. pymongo 4.17.0; macos ones are placeholders the upgrade-dependencies workflow will regenerate) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Python 3.14 changed the default multiprocessing start method on Linux from "fork" to "forkserver". Pool workers no longer inherit the parent's sys.modules, so they must re-import every pickled builder by name: - Builders defined in test modules (tests.cli.test_distributed) could not be imported because --import-mode=importlib leaves the repo root off sys.path when pytest is launched via its console script, as CI does. Add pythonpath = ["."] (pytest >= 7) so children can import them. - Builders loaded by `mrun` from a .py/.ipynb file are re-imported via ScriptFinder, which used regex.match even though `regex` is only installed with the notebook_runner extra. Use the stdlib re.match. Without these, every pool worker died with ModuleNotFoundError/NameError, AsyncUnorderedMap swallowed the resulting BrokenProcessPool, and multi() waited forever. The stuck non-daemon threads then kept the interpreter from exiting, so CI ran into the 6-hour job limit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Pt7k92GmQ4abuRUeW8CpQ
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1152 +/- ##
==========================================
+ Coverage 69.63% 69.77% +0.14%
==========================================
Files 47 47
Lines 4113 4106 -7
==========================================
+ Hits 2864 2865 +1
+ Misses 1249 1241 -8 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Supersedes #1151. It includes that PR's commit (
9782d35 Add Python 3.14 to supported versions, unchanged) plus a fix for the Python 3.14 CI hang that kept #1151 from going green.Root cause: Python 3.14 changed the default
multiprocessingstart method on Linux fromforktoforkserver. Pool workers no longer inherit the parent'ssys.modules, so they have to re-import every pickled builder by name. Two builder sources couldn't be imported:tests.cli.test_distributed).--import-mode=importlibleaves the repo root offsys.pathwhen pytest is launched through its console script, as CI does.mrunloads from a.py/.ipynbfile. These are re-imported throughScriptFinder, which usedregex.match.regexis only installed with thenotebook_runnerextra.When that happened, every pool worker died with
ModuleNotFoundError/NameError.AsyncUnorderedMapswallowed the resultingBrokenProcessPool, somulti()waited forever. The stuck non-daemon threads then kept the interpreter from exiting, and CI ran until it hit the 6-hour job limit.Major changes:
pyproject.toml: addpythonpath = ["."]to[tool.pytest.ini_options]so forkserver children can import builders defined in test modules. Bumpminversionto7.0, sincepythonpathneeds pytest 7 or later.src/maggma/cli/source_loader.py: use the stdlibre.matchinstead ofregex.match, soScriptFinderworks without thenotebook_runnerextra.Todos
test (ubuntu-latest, 3.14)andtest-min-deps (ubuntu-latest, 3.14).tests/cli/test_distributed.pyhas an intermittent fork-at-exit deadlock on Python 3.11–3.13. It reproduced in about 1 of 8 runs on unmodifiedmainand is caused by leakedProcessPoolExecutors inmulti(). If a 3.11–3.13 job hangs in that file, it is not a regression from this PR.Checklist
ruff. (pre-commit runpassed: ruff, black, codespell, file hygiene hooks)mypy. (n/a: no new signatures)tests/clitests cover the hang on 3.14)requirements/ubuntu-latest_py3.14_extras.txt,pytest tests/cli/test_distributed.py tests/cli/test_init.py::test_python_source --timeout 30(console script) gave 6 passed, 2 xfailed (pre-existing markers), and exited in about 12 s. MongoDB-backed tests were not run locally (no MongoDB available); they are left to CI.🤖 Generated with Claude Code
https://claude.ai/code/session_013Pt7k8oDSGSVZDB41JuKtk