Enables web scraping and crawling capabilities for Large Language Models.
Crawl4AI integrates web scraping and crawling as tools for Large Language Models, specifically within Cursor Composer's agent mode. It provides functionality for single page scraping with metadata extraction, as well as multi-page crawling with configurable depth and limits. The tool supports markdown and JSON output formats and offers a simple command-line interface for easy integration.
主要功能
01Single page scraping with metadata extraction
02Markdown and JSON output formats
03Simple command-line interface
04Easy integration with Cursor AI
051 GitHub stars
06Multi-page crawling with configurable depth and limits
使用案例
01Provides web scraping capabilities for Cursor AI