Airflow 2 to 3 Migration

关于

This skill facilitates the transition from Apache Airflow 2.x to 3.x by identifying breaking changes, refactoring legacy imports, and updating code to align with the new Airflow 3 architecture. It leverages Ruff migration rules for automated fixes and provides a comprehensive checklist for manual refactoring, such as replacing direct metadata database access with the Airflow Task Execution API and updating deprecated context variables. Ideal for data engineers looking to modernize their workflows while ensuring compatibility with Airflow 3’s decoupled scheduler and worker architecture.

主要功能

• Guidance on migrating Datasets to the new Asset-based system
• Identification and remediation of direct metadata database access issues
• Automated code refactoring using Ruff migration rules (AIR30/AIR31)
• Update mapping for legacy operators, hooks, and provider imports
• Verification of Airflow context keys and scheduling logic changes
• 77 GitHub stars

使用场景

• Upgrading legacy Airflow 2.x DAG repositories to the latest Airflow 3.x standard
• Refactoring codebase to eliminate deprecated imports and ensure future-proof data workflows
• Modernizing custom operators to use the new Task Execution API instead of ORM sessions