Skip to content

metro-file-map: Watch directories as the node crawler lists them - #2108

Draft
robhogan wants to merge 1 commit into
pr2107from
pr2108
Draft

robhogan wants to merge 1 commit into
pr2107from
pr2108

Conversation

@robhogan

@robhogan robhogan commented Oct 25, 2026 •

Copy link
Copy Markdown
Collaborator

Without Watchman, Watcher starts its backends before the crawl, so that nothing changed during the crawl is missed:

// Non-Watchman backends have no clock to subscribe `since`, so they must
// already be watching before the crawl begins, and their events buffered,
// for changes made while crawling not to be missed. Watchman backends are
// started by watch(), which subscribes `since` the crawl's clock to cover
// the same window. If the Watchman crawl falls back to the node crawler,
// there is no clock, so #crawl() starts non-Watchman backends before the
// node crawl instead.
if (!options.useWatchman) {
await this.#startBackendsBeforeCrawl();
}

For FallbackWatcher that means walking the tree once to watch it and again to crawl it. This lets the crawl do both: the crawler hands each directory to the watcher just before listing it, the order the watcher's own walk uses. Directories the crawl skips as ignored are handed over too, for the watcher to walk unless it ignores them as well (eg node_modules without retainAllFiles).

This is internal to the built-in node crawler. A crawlerFactory crawler, a recrawl and a root added with addRoot keep the watcher's own walk.

Benchmark

This takes another 28-30% off watch-mode startup on Linux and Windows. With the rest of this stack, watch-mode startup without Watchman goes from 1,896ms to 510ms (-73%) on Linux, and from 2,554ms to 522ms (-80%) on Windows. On macOS, which uses NativeWatcher, PRs 2-5 measure within noise of #2104.

main #2107 This PR vs main vs #2107
Linux: Watch-mode start, warm cache 1,896ms 707ms 510ms -73.1% (-73.4% to -72.8%) -27.9% (-28.6% to -27.2%)
Windows: Watch-mode start, warm cache 2,554ms 721ms 522ms -79.6% (-79.7% to -79.4%) -27.7% (-28.0% to -27.2%)
Linux: Watch-mode start, cold cache 1,882ms 699ms 491ms -73.9% (-74.5% to -73.5%) -29.7% (-31.1% to -28.5%)
Windows: Watch-mode start, cold cache 2,547ms 709ms 507ms -80.1% (-80.2% to -80.0%) -28.5% (-28.9% to -27.9%)
Linux: Heap after a watch-mode start 31.8MB 28.3MB 28.3MB -11.1% (-11.1% to -11.1%) -0.1% (-0.1% to -0.1%)
Windows: Heap after a watch-mode start 31.9MB 28.4MB 28.4MB -11.1% (-11.1% to -11.1%) -0.1% (-0.1% to -0.1%)
Linux: RSS after a watch-mode start 197.6MB 175.2MB 174.3MB -11.8% (-12.3% to -11.3%) -0.5% (-0.9% to +0.0%)
Windows: RSS after a watch-mode start 206.1MB 182.9MB 183.9MB -10.7% (-11.3% to -10.3%) +0.6% (+0.4% to +0.8%)

Changelog

 - **[Performance]**: metro-file-map: Set up file watching during the crawl instead of after it, on Linux and Windows without Watchman

Test plan

New unit tests, and a watch-integration-test.js case for a directory the crawl skips but the watcher doesn't. Benchmark as in #2104.

@robhogan
robhogan added this pull request to stack #2109 October 25, 2026 15:22
@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 25, 2026
@robhogan
robhogan force-pushed the pr2108 branch 2 times, most recently from b46060d to 7219809 Compare October 25, 2026 20:13
Without Watchman, `Watcher` starts its backends before the crawl, so that nothing changed during the crawl is missed:

https://github.com/react/metro/blob/56efab302200568c919ac1b3c28dacf8d84fc692/packages/metro-file-map/src/Watcher.js#L118-L127

For `FallbackWatcher` that means walking the tree once to watch it and again to crawl it. This lets the crawl do both: the crawler hands each directory to the watcher just before listing it, the order the watcher's own walk uses. Directories the crawl skips as ignored are handed over too, for the watcher to walk unless it ignores them as well (eg `node_modules` without `retainAllFiles`).

This is internal to the built-in node crawler. A `crawlerFactory` crawler, a recrawl and a root added with `addRoot` keep the watcher's own walk.

## Benchmark

This takes another 28-30% off watch-mode startup on Linux and Windows. With the rest of this stack, watch-mode startup without Watchman goes from 1,896ms to 510ms (-73%) on Linux, and from 2,554ms to 522ms (-80%) on Windows. On macOS, which uses `NativeWatcher`, PRs 2-5 measure within noise of #2104.

|  | `main` | #2107 | This PR | vs `main` | vs #2107 |
|---|---|---|---|---|---|
| Linux: Watch-mode start, warm cache | 1,896ms | 707ms | 510ms | -73.1% (-73.4% to -72.8%) | -27.9% (-28.6% to -27.2%) |
| Windows: Watch-mode start, warm cache | 2,554ms | 721ms | 522ms | -79.6% (-79.7% to -79.4%) | -27.7% (-28.0% to -27.2%) |
| Linux: Watch-mode start, cold cache | 1,882ms | 699ms | 491ms | -73.9% (-74.5% to -73.5%) | -29.7% (-31.1% to -28.5%) |
| Windows: Watch-mode start, cold cache | 2,547ms | 709ms | 507ms | -80.1% (-80.2% to -80.0%) | -28.5% (-28.9% to -27.9%) |
| Linux: Heap after a watch-mode start | 31.8MB | 28.3MB | 28.3MB | -11.1% (-11.1% to -11.1%) | -0.1% (-0.1% to -0.1%) |
| Windows: Heap after a watch-mode start | 31.9MB | 28.4MB | 28.4MB | -11.1% (-11.1% to -11.1%) | -0.1% (-0.1% to -0.1%) |
| Linux: RSS after a watch-mode start | 197.6MB | 175.2MB | 174.3MB | -11.8% (-12.3% to -11.3%) | -0.5% (-0.9% to +0.0%) |
| Windows: RSS after a watch-mode start | 206.1MB | 182.9MB | 183.9MB | -10.7% (-11.3% to -10.3%) | +0.6% (+0.4% to +0.8%) |

## Changelog

```
 - **[Performance]**: metro-file-map: Set up file watching during the crawl instead of after it, on Linux and Windows without Watchman
```

## Test plan

New unit tests, and a `watch-integration-test.js` case for a directory the crawl skips but the watcher doesn't. Benchmark as in #2104.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant