Can it document PII and security rules?

Yes, one of its core strengths is identifying PII hashing, masking, and security transformations within your SQL code to include them in the generated documentation.

What documentation format does it output?

It provides a highly structured Markdown report based on a production-tested template, which can be easily pasted into Confluence, GitHub, or other internal wikis.

Does it support incremental load documentation?

Yes, it analyzes the workflow files to distinguish between initial load and incremental load steps, documenting the logic for each phase separately.

Which SQL engines are supported?

It automatically detects and supports Presto, Trino, and Hive engines, adjusting the documentation style and syntax analysis based on the detected engine.

Does this skill work without codebase access?

No, this skill strictly requires access to your .dig, .sql, and .yml files to extract real transformation logic and ensure the documentation contains actual table names and SQL examples.

Staging Layer Documentation Expert

Name: Staging Layer Documentation Expert
Author: treasure-data

bytreasure-data

•

학습 및 문서화

Generates comprehensive documentation for staging transformation layers by analyzing SQL engines, PII handling, and deduplication logic directly from codebase files.

소개

This skill automates the creation of production-ready documentation for data staging workflows, specifically designed for environments using Presto, Trino, or Hive. It analyzes .dig workflows, .yml configurations, and .sql transformation files to extract real-world logic, including PII masking, date standardization, and complex deduplication strategies. By requiring direct codebase access, it ensures that generated guides are grounded in actual implementation details rather than generic placeholders, making it an essential tool for data engineers maintaining complex ETL/ELT pipelines and ensuring compliance through accurate documentation.

주요 기능

Extraction of real PII handling and masking logic from source code
Detailed analysis of deduplication strategies and data quality rules
Mapping of workflow phases and directory structures from .dig files
2 GitHub stars
Automatic SQL engine detection for Presto, Trino, and Hive
Support for documenting incremental and full load processing modes

사용 사례

Documenting complex SQL transformation logic for team handovers and onboarding
Creating standardized Confluence-ready documentation for existing data pipelines
Auditing PII handling and security transformations across staging layers

소개

주요 기능

Extraction of real PII handling and masking logic from source code
Detailed analysis of deduplication strategies and data quality rules
Mapping of workflow phases and directory structures from .dig files
2 GitHub stars
Automatic SQL engine detection for Presto, Trino, and Hive
Support for documenting incremental and full load processing modes

사용 사례

Documenting complex SQL transformation logic for team handovers and onboarding
Creating standardized Confluence-ready documentation for existing data pipelines
Auditing PII handling and security transformations across staging layers