Senior Data Engineer

关于

This skill equips Claude with the technical leadership and specialized knowledge required to architect, implement, and optimize enterprise-scale data systems. It provides production-ready guidance for building fault-tolerant distributed computing frameworks, orchestrating complex workflows, and integrating MLOps best practices. Whether you are designing a high-throughput real-time inference system or establishing robust DataOps and governance standards, this skill offers the advanced patterns and performance-tuning strategies necessary to ensure high availability and data quality for modern AI/ML environments.

主要功能

• Automated data quality validation and performance optimization strategies
• 310 GitHub stars
• Advanced data modeling and distributed architecture design principles
• Security and compliance frameworks for PII handling and GDPR/CCPA
• Enterprise-scale data pipeline orchestration using Airflow, dbt, and Spark
• Production DataOps and MLOps patterns for model deployment and monitoring

使用场景

• Deploying production ML models with feature store integration and drift detection
• Designing and implementing scalable ETL/ELT workflows for large-scale data processing
• Optimizing data infrastructure for high-throughput and low-latency performance targets