Codex plugin with the skills of the Scrapfly web data APIs: scraping with JavaScript rendering and proxy pools, screenshots, AI data extraction, crawling, Cloud Browser automation and alerting, through the scrapfly CLI and the Python, TypeScript, Go and Rust SDKs.
scrapfly-agent-rules: Cross-tool golden rules for an autonomous Scrapfly web agent connected to the Scrapfly MCP server (web_scrape, screenshot, the Cloud Browser lifecycle, snapshot/click/fill/type/press/scroll/select/drag, and WebMCP).scrapfly-alerting: Threshold-based alerting on Scrapfly account metrics.scrapfly-browser: Automate cloud browsers using the Scrapfly Cloud Browser API with Python Playwright.scrapfly-cli: Use the Scrapfly CLI (scrapfly) to scrape web pages, capture screenshots, extract structured data with AI, crawl entire sites, and drive a cloud browser over CDP from a shell.scrapfly-crawler: Crawl entire websites using the Scrapfly Crawler API with the Python SDKscrapfly-extraction: Extract structured data from web content using the Scrapfly Extraction API with the Python SDKscrapfly-scraper: Web scraping using the Scrapfly Scraper API with the Python SDKscrapfly-screenshot: Capture web page screenshots using the Scrapfly Screenshot API with the Python SDKscrapfly-sdk-go: Web scraping from Go with the Scrapfly SDK (github.com/scrapfly/go-scrapfly).scrapfly-sdk-rust: Web scraping from Rust with the Scrapfly SDK (scrapfly-sdkcrate).scrapfly-sdk-typescript: Web scraping from TypeScript or JavaScript with the Scrapfly SDK (scrapfly-sdk).
- A Scrapfly API key, from the Scrapfly dashboard, in
SCRAPFLY_API_KEY. - For the skills that shell out to it, the Scrapfly CLI (
scrapfly). - For the SDK skills, the SDK of the language: Python, TypeScript, Go, Rust.
Until the plugin is listed in the Codex plugin directory, install it from a local marketplace:
- Clone this repository, for example into
plugins/scrapfly-openai-pluginat the root of the repository you use Codex in. - Add
.agents/plugins/marketplace.jsonat that root, shaped likeexamples/marketplace.json:source.pathis the plugin folder, relative to the marketplace file and starting with./. - Restart Codex, open Plugins (or
/pluginsin the Codex CLI), pick the marketplace and install Scrapfly.
With the Codex CLI, codex plugin marketplace add ./path-to-marketplace-root registers the same marketplace.
In a new Codex thread:
- "Use Scrapfly to scrape https://web-scraping.dev/products and list each product's name and price."
- "Use Scrapfly to take a full-page screenshot of https://web-scraping.dev."
- "Use Scrapfly to crawl https://web-scraping.dev, 20 pages at most, and summarize the sections of the site."
MIT, see LICENSE.