perf(file_utils): import httpx lazily (BE-10537) - #830
Conversation
`file_utils` imported httpx at module top, and two module-level constants built httpx objects at import time. Only `_download_file_httpx`, `download_file` and `_friendly_network_error` need it; `knowledge.py`, `build.py`, `workflow.py`, `skills` and the rest import this module for `atomic_write_text` / `atomic_write_bytes` alone. `import httpx` moved into those three functions, and the timeout and exception tuples became `functools.cache` helpers built on first download. Timeout and retry behaviour is unchanged. `import comfy_cli.knowledge`: 79 ms -> 70 ms (min of 11, two alternating rounds). httpx cost 11 ms there and no longer appears in `-X importtime`. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Team Run ID: 📒 Files selected for processing (2)
Included review availability: Your plan provides up to 4 included reviews per hour; 2 remain after this review. 📝 WalkthroughWalkthroughChangesThe change defers Lazy httpx loading
Suggested reviewers: Merge Risk: ⚪ Minimal · up to This change defers the httpx dependency until network functionality is used, reducing import-time overhead while preserving download behavior. No actionable merge-blocking risk remains after normal checks and review. 🚥 Pre-merge checks | ✅ 2✅ Passed checks (2 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
✨ Simplify code
Comment |
skishore23
left a comment
There was a problem hiding this comment.
Approving. Verified the three things that matter for a lazy-import move: nothing outside file_utils.py references the removed names (_DOWNLOAD_TIMEOUT/_TRANSIENT_EXCEPTIONS/_RETRIABLE_EXCEPTIONS — the only grep hit is transfer.py's unrelated _DOWNLOAD_TIMEOUT_S); the functools.cache helpers return the same tuple object every call so except-clause behaviour is identical; and the aria2 early-return really does precede the import httpx in download_file. The -X importtime subprocess test pins the win. Ran test_file_utils_network.py + test_file_utils.py on the branch: 129 passed; CI fully green.
Linear: BE-10537. Follow-up to BE-9907.
Problem
comfy_cli/file_utils.pyimported httpx at module top, and two module-level constants built httpx objects at import time (_DOWNLOAD_TIMEOUT = httpx.Timeout(...)and the transient/retriable exception tuples). Only_download_file_httpx,download_fileand_friendly_network_errorneed httpx. Everything else that imports this module wantsatomic_write_textoratomic_write_bytesalone:knowledge.py,build.py,build_spec.py,outdated.py,project.py,workflow.py,cql/loader.py,jobs_state.py,skills/__init__.py.Change
import httpxmoved into those three functions. Indownload_fileit sits after the aria2 early return, so the aria2 path never imports it.The two constants became three
functools.cachehelpers built on first download:_download_timeout,_transient_exceptions,_retriable_exceptions. Same classes in the same order, and the cache meansexcept _retriable_exceptions() as exc:compares against the same tuple object every time, so timeout and retry behaviour is unchanged. Nothing outside the module referenced the removed names.atomic_write_text/atomic_write_byteswere not split into a new module, per the ticket.Measurements
python -X importtime -c "import comfy_cli.knowledge"on a quiet machine, dependencies fromuv sync --locked, min of 11, two alternating rounds againstorigin/main.comfy_cli.knowledgehttpxlineThe ticket estimated 0.13 s. The real figure on a warm machine is 11 ms, because
richis already loaded by the time httpx pullshttpx._main.Tests
uv run --locked --extra dev pytest, matching the CI workflow:origin/main: 7281 passed, 39 skippedThe new test is
test_knowledge_import_does_not_pull_httpxintests/test_file_utils_network.py. It runs-X importtimein a subprocess and asserts httpx is absent from the import list. Verified it fails againstorigin/main.Download behaviour is covered by the existing suites, which pass unchanged:
tests/test_file_utils_network.pyandtests/comfy_cli/command/test_model_download_background.py, 301 tests including the timeout and retry paths.uvx ruff@0.15.15 checkandformat --diffboth clean, matching.github/workflows/ruff_check.yml.🤖 Generated with Claude Code