Does this skill support Airflow 2.0 features?

Yes, it prioritizes modern Airflow 2.x patterns, specifically emphasizing the TaskFlow API (@dag and @task decorators) for simplified data passing and cleaner code.

Does it provide assistance with Airflow testing?

Yes, the skill provides standardized patterns for testing DAG integrity and loading using Pytest and the Airflow DagBag utility.

Can I implement custom alerting and error handling?

Yes, it includes templates for failure callbacks and trigger rules like ALL_DONE or NONE_FAILED to ensure your pipelines handle errors gracefully and notify teams via Slack or email.

Can I use this for complex task dependency management?

Absolutely. It includes implementation patterns for fan-in/fan-out structures, branching logic, and cross-DAG dependencies using ExternalTaskSensors.

Airflow DAG Patterns

Name: Airflow DAG Patterns
Author: wshobson

bywshobson

•

23,194

Productividad y Flujo de Trabajo

Builds production-grade Apache Airflow workflows using best-practice design patterns for task orchestration and data pipelines.

Acerca de

This skill equips Claude with the patterns and best practices needed to engineer robust Apache Airflow DAGs. It covers essential orchestration concepts such as idempotency, TaskFlow API implementation, dynamic DAG generation, and complex branching logic. Whether you are building an ETL pipeline, setting up S3 sensors, or implementing custom error-handling callbacks, this skill provides the implementation templates and testing strategies required for scalable, production-ready data engineering and workflow automation.

Características Principales

Modern TaskFlow API implementation for cleaner, decorator-based workflows
Advanced orchestration logic including branching, conditional execution, and trigger rules
23,194 GitHub stars
Dynamic DAG factory patterns for programmatically generating multiple pipelines
Robust sensor patterns for S3, APIs, and external task dependencies
Standardized error handling, retry policies, and notification callback templates

Casos de Uso

Architecting scalable ETL/ELT data pipelines with automated retries and idempotency
Migrating legacy PythonOperator workflows to the modern Airflow 2.x TaskFlow API
Implementing cross-DAG dependencies and automated data quality checks

Acerca de

Características Principales

Modern TaskFlow API implementation for cleaner, decorator-based workflows
Advanced orchestration logic including branching, conditional execution, and trigger rules
23,194 GitHub stars
Dynamic DAG factory patterns for programmatically generating multiple pipelines
Robust sensor patterns for S3, APIs, and external task dependencies
Standardized error handling, retry policies, and notification callback templates

Casos de Uso

Architecting scalable ETL/ELT data pipelines with automated retries and idempotency
Migrating legacy PythonOperator workflows to the modern Airflow 2.x TaskFlow API
Implementing cross-DAG dependencies and automated data quality checks