How does this skill handle service failures?

It implements 'unless-stopped' Docker restart policies and systemd 'always' restart configurations to ensure services automatically recover from crashes or server reboots.

What does the Scraper Wrapper include?

The wrapper is a Bun-based service that handles the Firecrawl API requests, saves the resulting markdown to a local directory, and returns a public URL for easy access.

Firecrawl is an open-source engine that crawls websites and converts them into clean markdown format specifically optimized for LLM use cases.

Why use a self-hosted Firecrawl instance?

Self-hosting provides full control over data privacy, removes external API costs, and allows you to customize scraping depth and rate limits for your specific needs.

Firecrawl Self-Hosted Manager

Name: Firecrawl Self-Hosted Manager
Author: terrylica

byterrylica

•

Web Scraping y Recopilación de Datos

Manages and troubleshoots self-hosted Firecrawl instances for high-performance web-to-markdown scraping.

A comprehensive skill for deploying, maintaining, and automating a self-hosted Firecrawl environment. It provides a robust architecture using Docker Compose, a custom Bun-based scraper wrapper for easier file management, and Caddy for serving scraped results. This skill specifically addresses common reliability issues like container restart policies and connectivity over ZeroTier, ensuring your AI-driven data collection pipeline remains operational and efficient for LLM-ready data extraction.

Características Principales

01Pre-configured health checks and troubleshooting routines for API, Redis, and Playwright services.

02Integrated Bun wrapper for converting web content directly to markdown files via a simple API.

03Automated deployment and configuration of the full Firecrawl stack via Docker Compose.

04Optimized restart policies and YAML anchors to ensure service high availability and resilience.

05Standardized systemd service management for Caddy file servers and scraper utilities.

069 GitHub stars

Casos de Uso

01Converting complex websites into clean markdown for LLM ingestion and RAG knowledge base building.

02Setting up a private web scraping infrastructure to bypass rate limits and costs of hosted services.

03Troubleshooting container failures and connectivity issues in distributed scraping clusters via ZeroTier.

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add terrylica/cc-skills firecrawl-self-hosted

For use in Claude.ai and ChatGPT

Download Skill