Why is my Firecrawl scrape returning empty markdown?

This skill helps diagnose if you are scraping JavaScript-heavy pages without sufficient rendering time and suggests the correct 'waitFor' parameters to fix it.

What is the most common mistake with Firecrawl's async crawl?

The most common mistake is assuming crawlUrl returns results immediately; this skill teaches you how to properly poll the job ID with backoff logic to retrieve data safely.

How does this skill help save Firecrawl credits?

It identifies missing limits and path filters in your code that prevent unintentional crawling of large sites, ensuring you only consume credits for the data you actually need.

Does this skill help with robots.txt compliance?

Yes, it provides implementation patterns that respect site constraints and rate limits to prevent IP bans and ensure ethical scraping practices.

Firecrawl Best Practices & Pitfall Prevention

Name: Firecrawl Best Practices & Pitfall Prevention
Author: micsapp

bymicsapp

0•

ウェブスクレイピングとデータ収集

Identifies and resolves common Firecrawl integration mistakes, anti-patterns, and resource management issues during code reviews.

The Firecrawl Known Pitfalls skill serves as a specialized auditor for web scraping workflows using the Firecrawl API. It helps developers avoid frequent mistakes such as incorrect async job polling, excessive credit consumption on large sites, and missing JavaScript rendering configurations. By providing actionable patterns for scraping limits, output formatting, and rate-limiting compliance, it ensures that your data collection pipelines are efficient, cost-effective, and robust against common scraping failures.

主な機能

01Configuration support for JavaScript-heavy Single Page Applications (SPAs)

02Compliance checks for robots.txt and rate-limiting best practices

03Detection of incorrect async job handling and polling logic

040 GitHub stars

05Validation of output format settings for Markdown and clean text

06Guidance on credit-saving strategies using limits and URL filters

ユースケース

01Auditing existing scraping code to reduce API credit waste and costs

02Debugging empty or malformed scrape results from JS-heavy websites

03Onboarding new developers to a project using Firecrawl for data collection

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add micsapp/micstec-skills firecrawl-known-pitfalls

For use in Claude.ai and ChatGPT

主な機能

01Configuration support for JavaScript-heavy Single Page Applications (SPAs)

02Compliance checks for robots.txt and rate-limiting best practices

03Detection of incorrect async job handling and polling logic

040 GitHub stars

05Validation of output format settings for Markdown and clean text

06Guidance on credit-saving strategies using limits and URL filters

ユースケース

01Auditing existing scraping code to reduce API credit waste and costs

02Debugging empty or malformed scrape results from JS-heavy websites

03Onboarding new developers to a project using Firecrawl for data collection

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add micsapp/micstec-skills firecrawl-known-pitfalls

For use in Claude.ai and ChatGPT