Which data warehouses does this skill support?

The generated code is warehouse-agnostic and supports popular destinations including ClickHouse, Snowflake, MotherDuck/DuckDB, and BigQuery.

Does it handle incremental loading?

Absolutely. The skill implements DLT resources with cursor fields and state management to ensure only new or updated data is fetched after the initial load.

What is DLT and why is it used here?

DLT (data-load-tools) is an open-source library that simplifies loading data from various sources into well-structured tables. This skill uses DLT to ensure your pipelines are robust, scalable, and easy to maintain.

Can I use this for existing projects?

Yes, the skill detects existing etl/ directories and skips the core framework setup, allowing you to focus on adding new sources or runners immediately.

How do I handle API rate limits?

The generated client.py files include patterns for authentication and rate limiting to prevent pipeline failures when communicating with external providers.

ETL Pipeline Builder

Name: ETL Pipeline Builder
Author: oxy-hq

byoxy-hq

Gestión de Bases de Datos

Builds and extends robust, maintainable ETL pipelines using DLT for API connectors and spreadsheet ingestion.

Acerca de

The ETL Pipeline Builder skill automates the creation of production-grade data pipelines using the DLT (data-load-tools) framework. It guides users through the entire development lifecycle—from initial project scaffolding and API client implementation to warehouse-agnostic transformations and automated backfills. By enforcing standardized directory structures and best practices like incremental loading and rate limiting, it ensures that data engineers and developers can quickly deploy reliable pipelines for platforms like Toast, Square, and Stripe, or automate complex spreadsheet ingestion workflows.

Características Principales

0 GitHub stars
Warehouse-agnostic code generation compatible with Snowflake, BigQuery, ClickHouse, and DuckDB.
Incremental loading and backfill logic implementation using DLT resources.
Automated scaffolding of standardized ETL directory structures and core framework files.
Built-in support for API connectors with integrated authentication and rate limiting.
Sophisticated spreadsheet and file ingestion templates for XLSX and CSV data.

Casos de Uso

Scaling an existing ETL project with new data sources while maintaining a consistent architecture.
Building a new data ingestion pipeline for third-party APIs like Stripe or Square.
Automating the extraction of business metrics from messy Excel spreadsheets into a data warehouse.

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add oxy-hq/skills oxy-etl-builder

For use in Claude.ai and ChatGPT

Download Skill

GitHub

Acerca de

Características Principales

0 GitHub stars
Warehouse-agnostic code generation compatible with Snowflake, BigQuery, ClickHouse, and DuckDB.
Incremental loading and backfill logic implementation using DLT resources.
Automated scaffolding of standardized ETL directory structures and core framework files.
Built-in support for API connectors with integrated authentication and rate limiting.
Sophisticated spreadsheet and file ingestion templates for XLSX and CSV data.

Casos de Uso

Scaling an existing ETL project with new data sources while maintaining a consistent architecture.
Building a new data ingestion pipeline for third-party APIs like Stripe or Square.
Automating the extraction of business metrics from messy Excel spreadsheets into a data warehouse.