Xcrawl
XCrawl is a web scraping API wrapper that fetches and extracts content from URLs. You get markdown, HTML, links, screenshots, or structured JSON extraction via prompts or schemas. It runs sync or async, handles JS rendering, and manages proxies and sessions. The skill reads your API key from a local config file and uses curl or node to hit the XCrawl REST endpoints. Each request burns credits, so you'll need an account at dash.xcrawl.com. It's built for single-page scraping by default, though XCrawl itself has separate APIs for crawling, mapping, and search that this skill doesn't directly wrap. Straightforward if you need to pull data from web pages without managing your own headless browser setup.
- skills.sh installs
- 420
- GitHub stars
- 449
- last updated
- Aug 21, 2026
Install
$ npx -y skills add xcrawl-api/xcrawl-skills --skill xcrawl --agent claude-codeInstalls into .claude/skills of the current project.
Files
XCrawl
Overview
This skill is the default XCrawl entry point when the user asks for XCrawl directly without naming a specific API or sub-skill. It currently targets single-page extraction through XCrawl Scrape APIs. Default behavior is raw passthrough: return upstream API response bodies as-is.
Routing Guidance
- If the user wants to extract one or more specific URLs, use this skill and default to XCrawl Scrape.
- If the user wants site URL discovery, prefer XCrawl Map APIs.
- If the user wants multi-page or site-wide crawling, prefer XCrawl Crawl APIs.
- If the user wants keyword-based discovery, prefer XCrawl Search APIs.
Required Local Config
Before using this skill, the user must create a local config file and write XCRAWL_API_KEY into it.
Path: ~/.xcrawl/config.json
{
"XCRAWL_API_KEY": "<your_api_key>"
}
Read API key from local config file only. Do not require global environment variables.
Credits and Account Setup
Sponsors
Promote your product hereReach 125k+ visitors monthlyPromote your product hereReach 125k+ visitors monthly
Agent CaffeineSponsoredClose your lid, come back to finished work
SnugSponsoredThe clipboard manager for the AI eraPromote your product hereReach 125k+ visitors monthly
WaxSponsoredBlock anything with supervision, without erasing your iPhonePromote your product hereReach 125k+ visitors monthly
knobsSponsoredSee your site on any phone, any day, in any language
Agent CaffeineSponsoredClose your lid, come back to finished workSnugSponsoredThe clipboard manager for the AI eraPromote your product hereReach 125k+ visitors monthlyWaxSponsoredBlock anything with supervision, without erasing your iPhonePromote your product hereReach 125k+ visitors monthlyknobsSponsoredSee your site on any phone, any day, in any languageDetails
- Repository
- github.com/xcrawl-api/xcrawl-skills
- Categories
- Backend & APIs, Code Review & Quality, AI & Agent Building
- First seen
- Jun 3, 2026
