The firecrawl-crawl skill enables developers and content extraction teams to bulk extract structured content from entire websites or specific site sections by following links up to configurable depth and path limits. It solves the problem of efficiently scraping multiple interdependent pages at scale, handling concurrent extraction, and managing credit consumption for large-scale data collection tasks. The skill is typically used as the final step in a workflow after searching, scraping individual pages, and mapping site structure to extract complete documentation sections, archives, or site-wide content.
npx skills add https://github.com/firecrawl/cli --skill firecrawl-crawl