What is the purpose of the discover-data skill?

It acts as a gateway that automatically identifies and loads specific data engineering sub-skills when you work on ETL, data pipelines, or stream processing tasks.

Can I load specific data skills manually?

Yes, the skill provides specific shell commands to load individual modules like etl-patterns or stream-processing as needed.

Which data frameworks does this skill support?

It includes patterns and best practices for Airflow, Kafka, Flink, Timely Dataflow, and Differential Dataflow.

How do I access the full data category details?

You can run the command 'cat ~/.claude/skills/data/INDEX.md' within the Claude Code terminal to see all available descriptions and workflows.

Does this skill help with data validation?

Yes, it includes specialized sub-skills for validating data schemas before processing to ensure pipeline integrity.

Data Pipeline & ETL Discovery

Name: Data Pipeline & ETL Discovery
Author: rand

byrand

•

数据库管理

Automatically activates specialized data engineering skills for ETL, batch processing, and real-time stream orchestration.

This gateway skill streamlines data development by automatically surfacing relevant sub-skills when working with ETL workflows, data pipelines, and distributed dataflow systems. It provides a structured entry point to advanced patterns like timely dataflow, stream-processing with Kafka or Flink, and complex pipeline orchestration, ensuring that Claude uses the most context-efficient practices for large-scale data manipulation and low-latency analytics. By acting as a central discovery hub, it enables progressive loading of documentation and patterns to keep your AI context focused and relevant to the specific data task at hand.

主要功能

01Access to 9 specialized data engineering sub-skills

02Support for advanced frameworks like Airflow and Timely Dataflow

0357 GitHub stars

04Automatic activation for ETL and data pipeline tasks

05Comprehensive patterns for batch and stream processing

06Progressive loading system to maintain context efficiency

使用场景

01Implementing real-time analytics pipelines using Kafka and streaming aggregations

02Designing and orchestrating complex multi-step data workflows with Airflow

03Building incremental computation systems with differential and timely dataflow

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add rand/cc-polymath discover-data

For use in Claude.ai and ChatGPT

Download Skill