Provides web scraping APIs for Bluesky, Substack, and Hacker News, returning structured JSON data with optional x402 micropayment support.
The Data Collector is a powerful web scraping server built with FastAPI, offering specialized APIs for extracting information from popular platforms like Bluesky, Substack, and Hacker News. It delivers structured JSON data, complete with engagement metrics, making it ideal for developers and data analysts. The service uniquely integrates x402 micropayments for pay-per-call access, alongside traditional API key authentication, and supports discovery via A2A Agent Card and MCP endpoints.
主要功能
01Scrape Bluesky posts, Substack newsletters, and Hacker News stories
02Returns structured JSON with engagement metrics
03Supports x402 micropayments ($0.05 USDC on Base per call)
04API key authentication for regular use
05A2A Agent Card and MCP discovery endpoints
060 GitHub stars
使用案例
01Gathering social media data for analysis
02Collecting news and article content programmatically
03Integrating scraped data into other applications via API
