Does this skill help with duplicate data?

Yes, it includes idempotency patterns that generate deterministic keys, ensuring that the same scraping operation isn't executed multiple times unnecessarily.

How does the circuit breaker help my scraping tasks?

It monitors for request failures and 'trips' the circuit to prevent your application from overwhelming a failing service, allowing the system to recover gracefully.

What libraries are required for these patterns?

The patterns are designed to work with popular Node.js libraries such as Opossum for circuit breakers and P-Queue for bulkhead management.

What is the FireCrawl Reliability Patterns skill?

It is a specialized capability for Claude Code that provides standardized implementation patterns for building fault-tolerant and resilient web scraping integrations using FireCrawl.

FireCrawl Reliability Patterns

Name: FireCrawl Reliability Patterns
Author: Brmbobo

byBrmbobo

0•

网络抓取与数据收集

Implements robust error handling and fault-tolerance patterns for stable FireCrawl web scraping integrations.

This skill provides a comprehensive suite of production-ready reliability patterns for FireCrawl, ensuring your web scraping and data extraction pipelines remain resilient under load or during service interruptions. It includes pre-configured templates for circuit breakers using Opossum, idempotency logic to prevent duplicate processing, bulkhead isolation for task prioritization, and sophisticated dead-letter queue (DLQ) management for graceful error recovery. Ideal for developers building enterprise-grade scrapers that require high availability and predictable behavior.

主要功能

01Graceful degradation with fallback caching and health monitoring

020 GitHub stars

03Bulkhead pattern for isolating and prioritizing critical tasks

04Idempotency key generation for deterministic scraping operations

05Hierarchical timeout configurations for various operation types

06Circuit breaker implementation to prevent cascading system failures

使用场景

01Handling FireCrawl rate limits and temporary API outages automatically

02Building mission-critical web scraping services with high uptime requirements

03Managing large-scale data extraction pipelines with prioritized processing

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add brmbobo/web2podcast firecrawl-reliability-patterns

For use in Claude.ai and ChatGPT

Download Skill