What version of Airflow does this skill support?

This skill focuses on modern Airflow 2.0+ best practices, specifically emphasizing the TaskFlow API, decorators, and updated provider packages.

Does it include testing strategies?

The skill provides patterns for DAG integrity testing and loading validation using pytest to ensure your code is production-ready before deployment.

How does it handle complex task dependencies?

It provides implementation patterns for fan-in/fan-out structures, BranchPythonOperators, and trigger rules like NONE_FAILED_MIN_ONE_SUCCESS for complex logic.

What is the recommended way to handle external dependencies?

It demonstrates using Sensors (like S3KeySensor) and ExternalTaskSensors with 'reschedule' mode to efficiently wait for external data or upstream tasks.

Can I use this for dynamic pipeline creation?

Yes, the skill includes a dedicated factory pattern for generating multiple similar DAGs programmatically from configuration files or metadata.

Apache Airflow DAG Patterns

Name: Apache Airflow DAG Patterns
Author: pur3v4d3r

bypur3v4d3r

•

デプロイメントとDevOps

Builds production-ready Apache Airflow pipelines using industry best practices for orchestration, reliability, and observability.

This skill empowers developers to architect robust data orchestration workflows using Apache Airflow. It provides a comprehensive library of implementation patterns, ranging from modern TaskFlow API usage and dynamic DAG generation to advanced branching logic and sophisticated error handling. Designed to ensure data pipelines are idempotent, atomic, and observable, it guides users through the complexities of sensor implementation, external dependencies, and automated testing to maintain high-quality data engineering standards in any production environment.

主な機能

01Dynamic DAG generation strategies for scaling repetitive pipeline structures

02Standardized DAG design principles including idempotency and atomicity

031 GitHub stars

04Advanced orchestration patterns including branching, sensors, and external triggers

05Modern TaskFlow API implementation for clean, Pythonic data passing

06Robust error handling frameworks with custom failure callbacks and trigger rules

ユースケース

01Implementing cross-DAG dependencies and event-driven data processing

02Architecting scalable ETL/ELT pipelines with complex task dependencies

03Automating the deployment and validation of production-grade workflow schedules

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add pur3v4d3r/pur3-pkb-codebase airflow-dag-patterns

For use in Claude.ai and ChatGPT

Download Skill