What storage tiers does this skill recommend?

It suggests a three-tier architecture: Raw (Immutable landing zone), Processed (Cleaned and validated Parquet), and Curated (Business-ready aggregates optimized for analytics).

Can it help with schema changes?

Yes, it guides you through schema evolution patterns, specifically recommending Apache Iceberg for native evolution support or versioned schemas for raw Parquet implementations.

How does it help with query performance?

It provides strategies for partition pruning, optimal file sizing (targeting 100MB-1GB), and efficient compression settings like ZSTD to minimize I/O and S3 request costs.

When should I use Iceberg over raw Parquet?

This skill recommends Iceberg when your workload requires ACID transactions, multiple concurrent writers, time travel capabilities, or frequent partition evolution.

Data Lake Architect

Name: Data Lake Architect
Author: EmilLindfors

byEmilLindfors

数据库管理

Designs scalable data lake and lakehouse architectures using optimized partitioning, storage tiers, and high-performance formats like Iceberg and Parquet.

关于

This skill empowers Claude to act as a senior data architect, providing prescriptive guidance on organizing large-scale data in cloud storage. It specializes in modern lakehouse patterns, helping users implement three-tier storage layouts (Bronze/Silver/Gold), select optimal partitioning strategies to avoid the 'small file problem,' and choose between raw Parquet and Apache Iceberg for transactional integrity. Whether you are building a new data platform or optimizing an existing one, it offers best practices for schema evolution, storage lifecycle policies, and high-performance data modeling.

主要功能

0 GitHub stars
Time-based, Multi-dimensional, and Hash partitioning strategies
Schema design patterns including wide tables and nested structures
Table format selection between Parquet and Apache Iceberg
Three-tier storage organization (Raw, Processed, Curated)
File sizing optimization (100MB-1GB target range)

使用场景

Migrating legacy data stores to a modern Iceberg lakehouse
Optimizing query performance by resolving partitioning and file size issues
Designing a new cloud-based data lake from scratch

关于

主要功能

0 GitHub stars
Time-based, Multi-dimensional, and Hash partitioning strategies
Schema design patterns including wide tables and nested structures
Table format selection between Parquet and Apache Iceberg
Three-tier storage organization (Raw, Processed, Curated)
File sizing optimization (100MB-1GB target range)

使用场景

Migrating legacy data stores to a modern Iceberg lakehouse
Optimizing query performance by resolving partitioning and file size issues
Designing a new cloud-based data lake from scratch