Does it support incremental loading?

Absolutely. It provides specific patterns for implementing incremental processing to ensure only new or updated data is handled, significantly improving pipeline performance.

How does it handle pipeline failures?

It focuses on designing workflows with robust error handling, retry logic, and integrated alerting systems to notify engineering teams of failures immediately.

What orchestration tools does this skill support?

While it provides specific code patterns for Apache Airflow DAGs, the underlying design principles for idempotency and error handling are applicable across various orchestration platforms.

Can it help with data quality?

Yes, the skill includes guidance on implementing data quality checks and validation steps within your transformation logic to ensure data integrity before loading.

ETL Designer

Name: ETL Designer
Author: armanzeroeight

byarmanzeroeight

•

Gestión de Bases de Datos

Designs robust and scalable ETL/ELT pipelines with built-in orchestration, error handling, and monitoring capabilities.

The ETL Designer skill equips Claude with the expertise to architect sophisticated data pipelines using modern best practices for data engineering. It specializes in Apache Airflow orchestration, implementing idempotent operations to ensure data consistency, and designing incremental processing logic to optimize performance. Whether you are building complex data workflows or simple transformation scripts, this skill provides the necessary patterns for error logging, alerting, and data quality checks required to maintain production-grade data integrity.

Características Principales

0119 GitHub stars

02Pipeline health monitoring and logging

03Automated error handling and alerting

04Idempotent operation design

05Airflow DAG architecture patterns

06Incremental data processing logic

Casos de Uso

01Building automated data warehouse ingestion pipelines

02Migrating legacy scripts into scalable, monitored ETL jobs

03Designing resilient data transformation workflows with Apache Airflow

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add armanzeroeight/fastagent-plugins etl-designer

For use in Claude.ai and ChatGPT

Download Skill