Can it handle JavaScript-heavy websites?

Yes, the skill includes patterns for using Playwright, which allows the agent to scrape sites that require browser rendering.

Is there a cost to run these scrapers?

No, the recommended stack uses GitHub Actions, Gemini's free tier, and Notion or Google Sheets to ensure the agent runs 100% free within standard usage limits.

How does the agent learn from my feedback?

The agent stores your decisions (likes/dislikes) in a feedback JSON file, which is then injected into the AI's prompt to bias future scoring and summaries.

What happens if I hit the Gemini API rate limit?

The skill implements a robust fallback chain that automatically tries different Gemini models and utilizes batch processing to stay within free tier quotas.

Data Scraper Agent

Name: Data Scraper Agent
Author: hieuck

byhieuck

0•

Web Scraping & Data Collection

Builds fully automated, AI-powered data collection agents that scrape, enrich, and store public data for free.

The Data Scraper Agent skill provides a comprehensive framework for building production-grade data pipelines within Claude Code. It automates the process of monitoring public websites, APIs, and RSS feeds, then uses the Gemini Flash LLM to score and summarize the findings. Designed for efficiency and cost-effectiveness, the skill utilizes a free stack—including GitHub Actions for scheduling and Notion or Supabase for storage—while incorporating a feedback loop that allows the agent to learn your preferences and improve its accuracy over time.

Key Features

010 GitHub stars

02Feedback-driven learning system that refines AI results over time

03Support for Notion, Sheets, and Supabase storage backends

04Automated scheduling via GitHub Actions CRON

05Multi-model fallback logic to bypass API rate limits

06AI-powered data enrichment and scoring with Gemini Flash

Use Cases

01Aggregating and summarizing news, GitHub releases, or sports stats into a database

02E-commerce price tracking and competitive analysis alerts

03Automated job board monitoring with AI relevance scoring against a resume

Key Features

010 GitHub stars

02Feedback-driven learning system that refines AI results over time

03Support for Notion, Sheets, and Supabase storage backends

04Automated scheduling via GitHub Actions CRON

05Multi-model fallback logic to bypass API rate limits

06AI-powered data enrichment and scoring with Gemini Flash

Use Cases

01Aggregating and summarizing news, GitHub releases, or sports stats into a database

02E-commerce price tracking and competitive analysis alerts

03Automated job board monitoring with AI relevance scoring against a resume