Advanced Progressive Web Scraper

소개

This skill provides a robust framework for retrieving web content through a progressive escalation strategy. It starts with lightweight methods like WebFetch and Curl, escalating to full Browser Automation and finally Bright Data's professional scraping infrastructure when facing advanced bot detection or CAPTCHAs. This ensures high success rates for scraping any URL while optimizing for speed and resource efficiency, ultimately delivering clean markdown content ready for analysis or processing within Claude Code.

주요 기능

• Built-in support for JavaScript rendering via Playwright
• Advanced bot detection and CAPTCHA bypass via Bright Data MCP
• Automatic fallback logic for failed requests
• Four-tier progressive escalation strategy
• Standardized output in clean Markdown format
• 81 GitHub stars

사용 사례

• Extracting content from Single Page Applications (SPAs) requiring JS execution
• Automating high-volume web research where reliability is critical
• Bypassing 403 errors and sophisticated anti-bot measures on protected domains