What storage tiers does this skill recommend?

It suggests a three-tier architecture: Raw (Immutable landing zone), Processed (Cleaned and validated Parquet), and Curated (Business-ready aggregates optimized for analytics).

Can it help with schema changes?

Yes, it guides you through schema evolution patterns, specifically recommending Apache Iceberg for native evolution support or versioned schemas for raw Parquet implementations.

How does it help with query performance?

It provides strategies for partition pruning, optimal file sizing (targeting 100MB-1GB), and efficient compression settings like ZSTD to minimize I/O and S3 request costs.

When should I use Iceberg over raw Parquet?

This skill recommends Iceberg when your workload requires ACID transactions, multiple concurrent writers, time travel capabilities, or frequent partition evolution.

Data Lake Architect

Name: Data Lake Architect
Author: EmilLindfors

byEmilLindfors

0•

データベース管理

Designs scalable data lake and lakehouse architectures using optimized partitioning, storage tiers, and high-performance formats like Iceberg and Parquet.

This skill empowers Claude to act as a senior data architect, providing prescriptive guidance on organizing large-scale data in cloud storage. It specializes in modern lakehouse patterns, helping users implement three-tier storage layouts (Bronze/Silver/Gold), select optimal partitioning strategies to avoid the 'small file problem,' and choose between raw Parquet and Apache Iceberg for transactional integrity. Whether you are building a new data platform or optimizing an existing one, it offers best practices for schema evolution, storage lifecycle policies, and high-performance data modeling.

主な機能

010 GitHub stars

02Time-based, Multi-dimensional, and Hash partitioning strategies

03Schema design patterns including wide tables and nested structures

04Table format selection between Parquet and Apache Iceberg

05Three-tier storage organization (Raw, Processed, Curated)

06File sizing optimization (100MB-1GB target range)

ユースケース

01Migrating legacy data stores to a modern Iceberg lakehouse

02Optimizing query performance by resolving partitioning and file size issues

03Designing a new cloud-based data lake from scratch

主な機能

010 GitHub stars

02Time-based, Multi-dimensional, and Hash partitioning strategies

03Schema design patterns including wide tables and nested structures

04Table format selection between Parquet and Apache Iceberg

05Three-tier storage organization (Raw, Processed, Curated)

06File sizing optimization (100MB-1GB target range)

ユースケース

01Migrating legacy data stores to a modern Iceberg lakehouse

02Optimizing query performance by resolving partitioning and file size issues

03Designing a new cloud-based data lake from scratch