AI skills for web scraping

Web scraping skills give an agent a repeatable way to fetch pages and turn them into structured data: product lists, article text, tables, links or metadata. The better ones also cover pagination, rate limits and what to do when a page needs JavaScript to render.

Before you install one, check the target site’s terms and robots.txt, and prefer an official API when one exists. The skills below differ mostly in their fetching backend (plain HTTP, a headless browser or a hosted crawling service), so choose based on the sites you need to reach.

Recommended skills

The highest-ranked of 55 matching skills, by quality score and popularity.

Browser automation CLI for AI agents. Use when the user needs to interact with websites, including navigating pages, filling forms, clicking buttons, taking screenshots, extracting data, testing web apps, or automating any browser task. Triggers include requests to "open a website", "fill out a form", "click a button", "take a screenshot", "scrape data from a page", "test this web app", "login to a site", "automate browser actions", or any task requiring programmatic web interaction.

↑ GrowingActive Claude Code
66k GitHub stars MIT Updated Sep 21, 2026 Free

Derive new Office files from structural selections without touching the original. Use when the user points at part of a spreadsheet, Word document, PDF, or PowerPoint deck (a worksheet range, paragraph, page, slide, shape, or a pasted selection-ref block) and wants it extracted, converted, or edited — the result is always a NEW file; the source file is never modified. Covers xlsx range extraction to csv/markdown/xlsx, docx paragraph extraction and text replacement, pdf page extraction, pptx slide/shape/table-cell extraction and editing, and targeted xlsx cell edits.

NewActive Claude CodeCodex
52k GitHub stars AGPL-3.0-only Updated Sep 21, 2026 Free

Fetch any X/Twitter post as clean LLM-friendly JSON. Converts x.com, twitter.com, or adhx.com links into structured data with full article content, author info, and engagement metrics. No scraping or browser required.

↑ GrowingActive Claude CodeCodexCursor+2
47k GitHub stars MIT Updated Sep 21, 2026 Free

Stop Codex, GitHub Copilot, Claude Code, and Cursor from shipping generic UI. Use UIZZE’s public catalogue of 800,000+ real web and iOS screens to extract product-specific design decisions and enforce a hard finish gate for web and iOS interfaces.

Official Claude CodeCodexCursor+1
38k GitHub stars MIT Updated Aug 18, 2026 Free

Web toolkit powered by Exa, tuned for scientific and technical content. Use this skill when the user needs to search the web or fetch/extract URL content. Covers: web search (semantic lookups, research, current info — with optional research-paper category and academic domain filtering) and URL extraction (fetching pages, articles, academic PDFs in batch). Use this skill for web-related tasks when the user wants high-quality search or scholarly filtering via category=research paper. Triggers on requests to search, look up, fetch a page, or extract an article.

34k GitHub stars MIT Updated Aug 17, 2026 Free

Add local office-document-to-Markdown conversion to NanoClaw agent containers with the pinned Firecrawl AnyDoc CLI. Use when agents need to read attached Word, PowerPoint, Excel, OpenDocument, RTF, EPUB, CSV, or text-based PDF files without uploading them to a hosted parser.

Claude Code
31k GitHub stars MIT Updated Aug 18, 2026 Free

Add Tavily Search and Extract as keyless remote MCP tools for selected NanoClaw agent groups. Use when installing Tavily web search or URL extraction without an API key.

Claude Code
31k GitHub stars MIT Updated Aug 18, 2026 Free

Use when the user asks to automate browser tasks, scrape websites, fill forms, capture screenshots, extract structured data from web pages, or build web automation workflows. NOT for testing — use playwright-pro for that.

Claude CodeCodexCursor+1
25k GitHub stars MIT Updated Aug 17, 2026 Free

Automatically detect source types and build AI skills using Skill Seekers. Use when the user wants to create skills from documentation, repos, PDFs, videos, or other knowledge sources.

15k GitHub stars MIT Updated Aug 9, 2026 Free

Also popular

More matching skills with the most GitHub stars.

Automate web scraping and data extraction with Apify -- run Actors, manage datasets, create reusable tasks, and retrieve crawl results through the Composio Apify integration.

Official↑ GrowingActive Claude CodeCodexCursor+2
75k GitHub stars No license Updated Sep 18, 2026 Free

Automates browser interactions for web testing, form filling, screenshots, and data extraction. Use when the user needs to navigate websites, interact with web pages, fill forms, take screenshots, test web applications, or extract information from web pages.

↑ GrowingActive CodexCursorOpenCode
69k GitHub stars Other Updated Aug 31, 2026 Free

Browser automation with persistent page state. Use when users ask to navigate websites, fill forms, take screenshots, extract web data, test web apps, or automate browser workflows. Trigger phrases include "go to [url]", "click on", "fill out the form", "take a screenshot", "scrape", "automate", "test the website", "log into", or any browser interaction request.

↑ GrowingActive CodexCursorOpenCode
69k GitHub stars Other Updated Aug 31, 2026 Free

Related

Custom skills

Need a web scraping skill for your own stack?

Tell us what you want your AI agent to do. We’ll help turn your workflow, standards or internal tools into a production-ready skill for Claude Code, Codex, Cursor or Gemini.

Frequently asked questions

Can an AI agent scrape websites on its own?
With a scraping skill installed, yes: the skill tells the agent which tool to call, how to paginate and what output format to produce. The agent still needs network access in its environment.
Is web scraping legal?
It depends on the site, the data and your jurisdiction. Respect robots.txt and terms of service, avoid personal data and prefer official APIs. None of the skills listed here grant permission to scrape a site.