Senior Data Engineer
Builds scalable, production-grade data pipelines and ETL/ELT systems using the modern data stack.
This skill equips Claude with the expertise of a senior data engineer to design, implement, and optimize enterprise-scale data architectures. It provides comprehensive guidance on building fault-tolerant ETL/ELT pipelines, managing distributed computing frameworks like Spark, and implementing DataOps practices for seamless data lifecycle management. Whether you are architecting real-time streaming systems with Kafka or modeling data warehouses with dbt and Snowflake, this skill ensures best practices in scalability, security, and performance are applied to every data infrastructure task.
Características Principales
011 GitHub stars
02End-to-end ETL/ELT pipeline orchestration with Airflow and dbt
03Real-time data streaming and high-throughput inference architectures
04DataOps integration for automated testing, monitoring, and quality assurance
05Distributed data processing strategies using Spark and Databricks
06Advanced data modeling patterns for scalable data warehousing
Casos de Uso
01Migrating legacy data workflows to modern distributed cloud infrastructure
02Designing high-availability data architectures for production AI/ML systems
03Implementing comprehensive data quality and observability frameworks