Which data orchestration tools are supported?

This skill provides specific implementation patterns for Apache Airflow, Dagster, and Prefect, covering both batch and event-driven orchestration.

How does it handle data quality?

It implements data quality through automated dbt tests, validation rules for completeness and accuracy, and integration with observability tools like Monte Carlo.

Is it useful for streaming data architectures?

Yes, the skill includes patterns for streaming processing using Kafka consumers and real-time data ingestion strategies.

Can this skill help with data warehouse optimization?

Yes, it includes specialized techniques for BigQuery and Snowflake, such as partitioning, clustering, and incremental materialization to reduce costs and improve query speed.

Data Engineering Expert

Name: Data Engineering Expert
Author: eyadsibai

byeyadsibai

数据库管理

Architects robust data pipelines, ETL/ELT workflows, and scalable data warehouse solutions using modern stack patterns.

关于

The Data Engineering skill transforms Claude into a specialized data architect capable of designing and implementing high-performance data infrastructures. It provides comprehensive guidance on the Modern Data Stack, covering everything from ingestion with Airbyte to transformation via dbt and orchestration through Airflow. Whether you are building complex Star Schemas, optimizing BigQuery partitions, or implementing automated data quality tests, this skill ensures your pipelines are idempotent, scalable, and maintainable.

主要功能

0 GitHub stars
Performance optimization for Snowflake, BigQuery, and S3/GCS
Data quality validation and observability framework implementation
Best-practice dbt model structuring (staging, intermediate, marts)
Production-ready Airflow DAG and Spark pipeline generation
Dimensional modeling and Star Schema design patterns

使用场景

Implementing automated data quality testing and monitoring systems
Building idempotent ETL/ELT pipelines for cloud data warehouses
Refactoring legacy SQL scripts into modular, version-controlled dbt projects

关于

主要功能

0 GitHub stars
Performance optimization for Snowflake, BigQuery, and S3/GCS
Data quality validation and observability framework implementation
Best-practice dbt model structuring (staging, intermediate, marts)
Production-ready Airflow DAG and Spark pipeline generation
Dimensional modeling and Star Schema design patterns

使用场景

Implementing automated data quality testing and monitoring systems
Building idempotent ETL/ELT pipelines for cloud data warehouses
Refactoring legacy SQL scripts into modular, version-controlled dbt projects