Does this skill require access to my code?

Yes, codebase access is mandatory. The skill must be able to read your .dig and .yml configuration files to extract real table names and connector logic.

Does it expose sensitive information like API keys?

No. The skill is designed to document the authentication pattern and lifecycle (e.g., OAuth 2.0 flow) without exposing actual secrets or credentials.

What types of data sources can it document?

It supports a wide range of sources including REST APIs, Databases (JDBC/SQL), File-based storage (S3/GCS/SFTP), and Streaming services like Kafka or Kinesis.

Can it document incremental loading strategies?

Yes, it automatically detects and documents timestamp-based, sequence-based, and full reload patterns by analyzing your workflow's comparison logic.

APS Ingestion Documentation Expert

Name: APS Ingestion Documentation Expert
Author: treasure-data

bytreasure-data

•

学习与文档

Automatically generates detailed technical documentation for data ingestion layers by analyzing workflow configurations and connector patterns.

关于

The APS Ingestion Documentation Expert is a specialized skill designed to bridge the gap between complex data ingestion codebases and clear, production-ready documentation. By scanning directory paths for .dig and .yml files, it identifies connector types—including REST APIs, Databases, Cloud Storage, and Streaming services—to build structured, professional guides. This tool is essential for data engineering teams who need to maintain accurate, up-to-date documentation for data pipelines, authentication methods, and incremental load strategies without the manual overhead of tracking every configuration change.

主要功能

Automatic detection of REST API, Database, File, and Streaming connectors
Analysis of rate limiting, backoff strategies, and concurrent request limits
Secure identification and documentation of authentication patterns like OAuth 2.0 and JWT
Standardized template generation for parent-child documentation structures
Extraction of incremental load logic, watermark tracking, and sequence-based patterns
2 GitHub stars

使用场景

Building data flow diagrams and configuration references for S3 or GCS file-based ingestion
Generating comprehensive documentation for REST API connectors like Salesforce or HubSpot
Creating technical guides for database ingestion workflows from MySQL, PostgreSQL, or BigQuery

关于

主要功能

Automatic detection of REST API, Database, File, and Streaming connectors
Analysis of rate limiting, backoff strategies, and concurrent request limits
Secure identification and documentation of authentication patterns like OAuth 2.0 and JWT
Standardized template generation for parent-child documentation structures
Extraction of incremental load logic, watermark tracking, and sequence-based patterns
2 GitHub stars

使用场景

Building data flow diagrams and configuration references for S3 or GCS file-based ingestion
Generating comprehensive documentation for REST API connectors like Salesforce or HubSpot
Creating technical guides for database ingestion workflows from MySQL, PostgreSQL, or BigQuery