Does this skill support Airflow 2.0 features?

Yes, it prioritizes modern Airflow 2.x patterns, specifically emphasizing the TaskFlow API (@dag and @task decorators) for simplified data passing and cleaner code.

Does it provide assistance with Airflow testing?

Yes, the skill provides standardized patterns for testing DAG integrity and loading using Pytest and the Airflow DagBag utility.

Can I implement custom alerting and error handling?

Yes, it includes templates for failure callbacks and trigger rules like ALL_DONE or NONE_FAILED to ensure your pipelines handle errors gracefully and notify teams via Slack or email.

Can I use this for complex task dependency management?

Absolutely. It includes implementation patterns for fan-in/fan-out structures, branching logic, and cross-DAG dependencies using ExternalTaskSensors.

Airflow DAG Patterns

Name: Airflow DAG Patterns
Author: wshobson

bywshobson

•

23,194

生产力与工作流

Builds production-grade Apache Airflow workflows using best-practice design patterns for task orchestration and data pipelines.

关于

This skill equips Claude with the patterns and best practices needed to engineer robust Apache Airflow DAGs. It covers essential orchestration concepts such as idempotency, TaskFlow API implementation, dynamic DAG generation, and complex branching logic. Whether you are building an ETL pipeline, setting up S3 sensors, or implementing custom error-handling callbacks, this skill provides the implementation templates and testing strategies required for scalable, production-ready data engineering and workflow automation.

主要功能

Modern TaskFlow API implementation for cleaner, decorator-based workflows
Advanced orchestration logic including branching, conditional execution, and trigger rules
23,194 GitHub stars
Dynamic DAG factory patterns for programmatically generating multiple pipelines
Robust sensor patterns for S3, APIs, and external task dependencies
Standardized error handling, retry policies, and notification callback templates

使用场景

Architecting scalable ETL/ELT data pipelines with automated retries and idempotency
Migrating legacy PythonOperator workflows to the modern Airflow 2.x TaskFlow API
Implementing cross-DAG dependencies and automated data quality checks

关于

主要功能

Modern TaskFlow API implementation for cleaner, decorator-based workflows
Advanced orchestration logic including branching, conditional execution, and trigger rules
23,194 GitHub stars
Dynamic DAG factory patterns for programmatically generating multiple pipelines
Robust sensor patterns for S3, APIs, and external task dependencies
Standardized error handling, retry policies, and notification callback templates

使用场景

Architecting scalable ETL/ELT data pipelines with automated retries and idempotency
Migrating legacy PythonOperator workflows to the modern Airflow 2.x TaskFlow API
Implementing cross-DAG dependencies and automated data quality checks