Does it support Python-based data processing?

Absolutely. It includes specific instructions for data transformations using pandas and Spark, focusing on type safety and explicit NULL handling.

What is meant by 'idempotent' pipelines?

The skill ensures that data scripts are designed to be run multiple times without causing duplicate data or errors, which is a critical standard for reliable data engineering.

Can this skill help with database migrations?

Yes, it provides structured guidance on schema design and SQL best practices that are essential for planning and executing safe database migrations.

How does it improve SQL performance?

The skill enforces the use of specific column selection, prevents N+1 query problems, and utilizes Common Table Expressions (CTEs) to improve both readability and execution efficiency.

Can I use this for non-relational data?

While the core instructions highlight SQL and relational structures, the transformation and validation logic are highly applicable to JSON, NoSQL, and general data analysis tasks.

Data Engineering & SQL Optimizer

Name: Data Engineering & SQL Optimizer
Author: k1lgor

byk1lgor

0•

数据库管理

Builds efficient database schemas, optimized SQL queries, and robust ETL pipelines with a focus on data integrity and performance.

The Data Engineer skill transforms Claude into a specialized data architect capable of handling complex database design, high-performance SQL generation, and scalable data transformations. It provides rigorous workflows for normalizing schemas, optimizing query execution via CTEs and indexing, and building idempotent ETL pipelines using modern tools like pandas and Spark. By enforcing best practices like explicit NULL handling and constraint validation, this skill ensures that your data infrastructure remains efficient, type-safe, and production-ready.

主要功能

01Idempotent ETL pipeline development for data reliability

02Comprehensive data validation and transformation logic

030 GitHub stars

04Advanced database schema design and normalization

05Automated indexing strategies for query performance

06High-performance SQL optimization and CTE implementation

使用场景

01Designing a scalable relational database schema for complex applications

02Refactoring legacy SQL queries to eliminate performance bottlenecks

03Developing automated data cleaning and transformation scripts in pandas

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add k1lgor/virtual-company 11-data-engineer

For use in Claude.ai and ChatGPT

Download Skill