Why does this skill need to kill my Edge browser processes?

To enable remote debugging via CDP, Edge must be launched with specific flags. If an instance is already running, it will ignore these flags and join the existing process, so a fresh launch is required to enable automation.

What output formats does the scraper provide?

The scraper generates clean text (.txt) for content analysis, full source (.html) for structure, and a JSON inventory of all discovered links for navigation mapping.

Does this work with sites requiring hardware keys or 2FA?

Yes. Because the skill launches a headed browser window, you can manually complete any 2FA, SSO, or hardware-based login (like YubiKey) before the automated scraping begins.

Can I use this in a standard Linux environment?

This specific implementation is optimized for WSL2 (Windows Subsystem for Linux), as it handles the specific networking and filesystem complexities of communicating between WSL and the Windows host.

Authenticated Web Scraper

Name: Authenticated Web Scraper
Author: rysweet

byrysweet

•

网络抓取与数据收集

Extracts content from password-protected websites and internal documentation sites by leveraging the Windows Edge browser via Chrome DevTools Protocol.

This skill enables Claude to scrape websites protected by 2FA, SSO, or corporate logins by bridging the gap between WSL2 and Windows host environments. It launches a headed Edge browser instance to allow for manual user authentication and then automates deep content extraction using the Chrome DevTools Protocol (CDP), making it an essential tool for mirroring internal wikis, corporate documentation, or private web portals that standard headless scrapers cannot access.

主要功能

01Native Chrome DevTools Protocol (CDP) integration for robust automation

02Seamless WSL2 to Windows host communication for browser control

03Extracts high-fidelity clean text, full HTML, and structured link maps

04Supports 2FA, SSO, and corporate login via headed Edge browser sessions

0530 GitHub stars

06Automated navigation crawling with configurable depth and link discovery

使用场景

01Extracting documentation from private sites requiring SAML or FIDO2 authentication

02Mirroring internal corporate wikis or Confluence pages for AI-driven analysis

03Scraping structured data from web portals that block standard headless automation

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add rysweet/amplihack authenticated-web-scraper

For use in Claude.ai and ChatGPT

Download Skill