docs: Add cookies management guide - #2097
Conversation
There was a problem hiding this comment.
Pull request overview
This PR adds a new “Cookie management” guide to the Python Crawlee docs, aimed at clarifying how cookies are stored on sessions and how to keep cookies/session state across retries (addressing #2060). It also cross-links the new guide from existing session, headers, and login documentation.
Changes:
- Added a new
Cookie managementguide covering reading/setting cookies, seeding cookies in new sessions, Playwright cookie syncing, retry patterns, and persistence across runs. - Added runnable Python code examples for the cookie guide (read/write, initial cookies, Playwright sync, retry strategies, persistence).
- Added references to the new cookie guide from existing guides (session management, HTTP headers, logging in).
Reviewed changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| docs/guides/session_management.mdx | Adds a cross-link to the new cookie guide from session management. |
| docs/guides/http_headers.mdx | Links cookie header discussion to the new cookie guide. |
| docs/guides/crawler_login.mdx | Links authentication/session guidance to the new cookie guide. |
| docs/guides/cookie_management.mdx | New guide documenting cookie behavior, retries, Playwright syncing, and persistence. |
| docs/guides/code_examples/cookie_management/read_write_cookies.py | Example for reading and setting session cookies. |
| docs/guides/code_examples/cookie_management/initial_cookies.py | Example for seeding cookies via create_session_settings. |
| docs/guides/code_examples/cookie_management/playwright_cookies.py | Example showing Playwright ↔ session cookie syncing behavior. |
| docs/guides/code_examples/cookie_management/retry_single_session.py | Example for preserving cookies across retries with a single-session pool. |
| docs/guides/code_examples/cookie_management/retry_pinned_session.py | Example for “retrying” by re-enqueueing a request pinned to a session. |
| docs/guides/code_examples/cookie_management/retry_restore_cookies.py | Example for restoring cookies via use_state + pre_navigation_hook. |
| docs/guides/code_examples/cookie_management/persist_cookies.py | Example for persisting cookies across runs via session pool persistence. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
21351a7 to
3ff1281
Compare
vdusek
left a comment
There was a problem hiding this comment.
LGTM, the text and examples are great, thank you.
Leaving here a few comments from Claude to consider, once they are resolved, I'll merge it (you can either address them or leave it as it is and just mark as resolved, up to you, they are mostly nits).
Co-authored-by: Vlada Dusek <v.dusek96@gmail.com>
Description
Issues