Is it compatible with orchestration tools?

Yes, the patterns are designed to be used within orchestration frameworks like Airflow, Dagster, or dbt for post-load transformations.

Does this skill support SQLAlchemy 2.0?

Yes, it specifically focuses on SQLAlchemy 2.0+ patterns including Mapped types, TypeDecorators, and modern repository patterns.

How does it handle API schema changes?

The skill provides patterns for schema resilience using PostgreSQL JSONB columns to capture evolving API fields without breaking existing database migrations.

Can I use this for dimensional modeling?

Absolutely. It includes best practices for Kimball star schemas, including fact/dimension naming conventions and bridge table patterns.

Python Data Engineering

Name: Python Data Engineering
Author: AeyeOps

byAeyeOps

Database Management

Builds production-grade data pipelines and dimensional models using SQLAlchemy 2.0 and modern warehouse architectures.

About

This skill provides comprehensive patterns for architecting robust Python-based data pipelines, focusing on SQLAlchemy 2.0 implementations and modern data warehousing principles like Kimball star schemas and medallion layers. It guides developers through implementing complex ETL/ELT workflows, managing schema evolution via JSONB, and ensuring data integrity with Pydantic validation. By establishing a clear three-layer separation between source systems, database schemas, and application logic, it enables the creation of maintainable, scalable, and high-performance data infrastructure on PostgreSQL.

Key Features

SQLAlchemy 2.0 advanced patterns including TypeDecorators and factory methods
Schema resilience techniques using PostgreSQL JSONB for evolving SaaS API data
Implementation of medallion architecture layers (bronze, silver, and gold)
0 GitHub stars
Dimensional modeling for Kimball star schemas using dim_ and fact_ conventions
Incremental sync strategies using high-water mark patterns for optimized performance

Use Cases

Building custom Python ETL pipelines for sources lacking standard connectors
Converting raw SaaS API data into structured PostgreSQL dimensional models
Implementing historical data tracking using Slowly Changing Dimensions (SCD Type 2)

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add aeyeops/aeo-skill-marketplace python-data-engineering

For use in Claude.ai and ChatGPT

Download Skill

GitHub