What is the Data Engineering skill?

It is a specialized Claude Code capability that provides best practices, architectural patterns, and implementation logic for building reliable and reproducible data workflows.

What tools are emphasized in this skill?

It focuses on modern engineering tools like uv for fast dependency management, Docker for containerization, and orchestration platforms such as Airflow, Prefect, and Dagster.

Does it support ELT patterns?

Yes, it includes specific guidance for both traditional ETL and modern ELT architectures, including dbt integration for in-warehouse transformations.

Can it help with data documentation?

Yes, it provides standardized templates for column-level documentation, data contracts, and lineage tracking to ensure data is understandable and traceable.

How does it handle data quality?

The skill incorporates schema validation via Pydantic and integrates with data testing frameworks like Great Expectations and dbt tests to ensure data integrity.

Data Engineering & Pipeline Optimization

Name: Data Engineering & Pipeline Optimization
Author: chekos

bychekos

0•

Data Science & ML

Implements reliable, reproducible data pipelines and infrastructure using best practices for ETL/ELT workflows.

This skill empowers Claude to architect and maintain production-grade data engineering workflows with a focus on reliability and reproducibility. It provides specialized guidance on building idempotent pipelines, managing data quality through schema validation and automated testing, and ensuring deterministic environments with modern tools like uv and dbt. Whether you are transitioning from ETL to ELT, setting up containerized data environments, or documenting complex data lineage, this skill ensures your data infrastructure is accessible, trustworthy, and scalable.

Key Features

01Optimizes dependency management and containerization for data workflows

02Provides reusable patterns for cross-platform data extraction and loading

03Architects idempotent ETL and ELT data pipelines

040 GitHub stars

05Standardizes data lineage documentation and schema validation

06Implements robust data quality testing with Great Expectations and dbt

Use Cases

01Building a reproducible data ingestion pipeline from APIs to a data warehouse

02Implementing automated data quality checks and schema validation for existing datasets

03Migrating legacy ETL scripts to modern ELT architectures using dbt and SQLMesh

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add chekos/bns-marketplace data-engineering

For use in Claude.ai and ChatGPT

Download Skill