Skip to content

Add Python 3.14 support (fix forkserver CI hang) - #1152

Merged
rkingsbury merged 2 commits into
materialsproject:mainfrom
rkingsbury:claude/gifted-feynman-w9v7ws
Oct 27, 2026
Merged

rkingsbury merged 2 commits into
materialsproject:mainfrom
rkingsbury:claude/gifted-feynman-w9v7ws

Conversation

@rkingsbury

Copy link
Copy Markdown
Collaborator

Summary

Supersedes #1151. It includes that PR's commit (9782d35 Add Python 3.14 to supported versions, unchanged) plus a fix for the Python 3.14 CI hang that kept #1151 from going green.

Root cause: Python 3.14 changed the default multiprocessing start method on Linux from fork to forkserver. Pool workers no longer inherit the parent's sys.modules, so they have to re-import every pickled builder by name. Two builder sources couldn't be imported:

  • Builders defined in test modules (tests.cli.test_distributed). --import-mode=importlib leaves the repo root off sys.path when pytest is launched through its console script, as CI does.
  • Builders that mrun loads from a .py/.ipynb file. These are re-imported through ScriptFinder, which used regex.match. regex is only installed with the notebook_runner extra.

When that happened, every pool worker died with ModuleNotFoundError/NameError. AsyncUnorderedMap swallowed the resulting BrokenProcessPool, so multi() waited forever. The stuck non-daemon threads then kept the interpreter from exiting, and CI ran until it hit the 6-hour job limit.

Major changes:

  • fix 1: pyproject.toml: add pythonpath = ["."] to [tool.pytest.ini_options] so forkserver children can import builders defined in test modules. Bump minversion to 7.0, since pythonpath needs pytest 7 or later.
  • fix 2: src/maggma/cli/source_loader.py: use the stdlib re.match instead of regex.match, so ScriptFinder works without the notebook_runner extra.

Todos

  • Watch the two 3.14 jobs, test (ubuntu-latest, 3.14) and test-min-deps (ubuntu-latest, 3.14).
  • Known pre-existing issue, not addressed here: tests/cli/test_distributed.py has an intermittent fork-at-exit deadlock on Python 3.11–3.13. It reproduced in about 1 of 8 runs on unmodified main and is caused by leaked ProcessPoolExecutors in multi(). If a 3.11–3.13 job hangs in that file, it is not a regression from this PR.

Checklist

  • Google format doc strings added. (n/a: no new functions)
  • Code linted with ruff. (pre-commit run passed: ruff, black, codespell, file hygiene hooks)
  • Type annotations included. Check with mypy. (n/a: no new signatures)
  • Tests added for new features/fixes. (existing tests/cli tests cover the hang on 3.14)
  • I have run the tests locally and they passed. On Python 3.14.6 with requirements/ubuntu-latest_py3.14_extras.txt, pytest tests/cli/test_distributed.py tests/cli/test_init.py::test_python_source --timeout 30 (console script) gave 6 passed, 2 xfailed (pre-existing markers), and exited in about 12 s. MongoDB-backed tests were not run locally (no MongoDB available); they are left to CI.

🤖 Generated with Claude Code

https://claude.ai/code/session_013Pt7k8oDSGSVZDB41JuKtk

rkingsbury and others added 2 commits October 26, 2026 17:30
Re-enables Python 3.14 support, which was previously added alongside the
3.9/3.10 drop (8c1e3c6) and then removed from CI in 942efbc pending
diagnosis of a CI hang. The 3.11-3.14 source compatibility fixes made on
that branch (asyncio event-loop, generator pattern, datetime.utcnow)
remain in place on main.

- pyproject.toml: add the Python 3.14 classifier (requires-python is
  already >=3.11, so no bound change needed)
- testing.yml: add 3.14 to both the `test` and `test-min-deps` matrices
- upgrade-dependencies.yml: add 3.14 so pinned requirements regenerate
- requirements: restore the py3.14 lock files (ubuntu ones are real
  cp314-compatible pins, e.g. pymongo 4.17.0; macos ones are placeholders
  the upgrade-dependencies workflow will regenerate)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Python 3.14 changed the default multiprocessing start method on Linux
from "fork" to "forkserver". Pool workers no longer inherit the parent's
sys.modules, so they must re-import every pickled builder by name:

- Builders defined in test modules (tests.cli.test_distributed) could not
  be imported because --import-mode=importlib leaves the repo root off
  sys.path when pytest is launched via its console script, as CI does.
  Add pythonpath = ["."] (pytest >= 7) so children can import them.
- Builders loaded by `mrun` from a .py/.ipynb file are re-imported via
  ScriptFinder, which used regex.match even though `regex` is only
  installed with the notebook_runner extra. Use the stdlib re.match.

Without these, every pool worker died with ModuleNotFoundError/NameError,
AsyncUnorderedMap swallowed the resulting BrokenProcessPool, and multi()
waited forever. The stuck non-daemon threads then kept the interpreter
from exiting, so CI ran into the 6-hour job limit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013Pt7k92GmQ4abuRUeW8CpQ
@codecov

codecov Bot commented Oct 27, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 69.77%. Comparing base (9a98aef) to head (0d144b4).
⚠️ Report is 30 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1152      +/-   ##
==========================================
+ Coverage   69.63%   69.77%   +0.14%     
==========================================
  Files          47       47              
  Lines        4113     4106       -7     
==========================================
+ Hits         2864     2865       +1     
+ Misses       1249     1241       -8     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@rkingsbury rkingsbury added dependencies Pull requests that update a dependency file pkg Package and repository health labels Oct 27, 2026
@rkingsbury
rkingsbury merged commit c07ace8 into materialsproject:main Oct 27, 2026
14 checks passed
@rkingsbury
rkingsbury deleted the claude/gifted-feynman-w9v7ws branch October 27, 2026 15:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file pkg Package and repository health

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants