Skip to content

test: mark test_get_snapshot with run_alone to deflake CI - #2150

Open
vdusek wants to merge 1 commit into
masterfrom
test/deflake-playwright-get-snapshot
Open

test: mark test_get_snapshot with run_alone to deflake CI#2150
vdusek wants to merge 1 commit into
masterfrom
test/deflake-playwright-get-snapshot

Conversation

@vdusek

@vdusek vdusek commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator

test_get_snapshot failed on macos-latest / Python 3.10 in run 31388378733 on assert snapshot.screenshot is not None, with Chromium answering Protocol error (Page.captureScreenshot): Unable to capture screenshot.

The macOS image is the smallest of the three (~3 vCPU / 7 GB), yet the -m "not run_alone" pass drives 8 xdist workers. At the failure timestamp three of them were each running their own Chromium: gw7 in test_get_snapshot, gw0 cycling the test_chromium_headless_headers parametrizations, and gw4 in the stagehand browser-controller tests. Under that contention the renderer cannot composite a frame for a full_page capture, and Playwright does not retry the error.

PlaywrightPreNavCrawlingContext.get_snapshot() deliberately degrades that failure to screenshot=None - an error snapshot must never break a crawl, and ErrorSnapshotter already guards on it - so the test observes None. The nondeterminism is runner contention, not a race in Crawlee, so product code is unchanged.

@pytest.mark.run_alone moves the test into the serial --numprocesses=1 pass, removing the contention while keeping every assertion intact. This is the second tier of the hierarchy documented in tests/unit/README.md; the same file already marks test_firefox_headless_headers and test_isolation_cookies this way.

The failure does not reproduce on Linux - 0/20 runs of the Playwright module at -n 16 (oversubscribing 8 cores) - so macOS CI is what re-verifies the fix. Locally the marker routes the test correctly (53 collected in the parallel pass, 4 in the serial one) and the full run_alone group is green at 14 items.

✍️ Drafted by Claude Code

@vdusek vdusek added t-tooling Issues with this label are in the ownership of the tooling team. adhoc Ad-hoc unplanned task added during the sprint. labels Aug 11, 2026
@vdusek vdusek self-assigned this Aug 11, 2026
@github-actions github-actions Bot added this to the 147th sprint - Tooling team milestone Aug 11, 2026
@github-actions github-actions Bot added the tested Temporary label used only programatically for some analytics. label Aug 11, 2026
@codecov

codecov Bot commented Aug 11, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.57%. Comparing base (165223e) to head (a300a7d).
⚠️ Report is 2 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2150      +/-   ##
==========================================
+ Coverage   93.55%   93.57%   +0.01%     
==========================================
  Files         181      181              
  Lines       12648    12648              
==========================================
+ Hits        11833    11835       +2     
+ Misses        815      813       -2     
Flag Coverage Δ
unit 93.57% <ø> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@vdusek vdusek changed the title test: mark test_get_snapshot with run_alone to deflake macOS CI test: mark test_get_snapshot with run_alone to deflake CI Aug 11, 2026
@vdusek
vdusek marked this pull request as ready for review August 11, 2026 07:30
@vdusek
vdusek requested a review from Pijukatel August 11, 2026 07:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

adhoc Ad-hoc unplanned task added during the sprint. t-tooling Issues with this label are in the ownership of the tooling team. tested Temporary label used only programatically for some analytics.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants