What is the difference between a Stream and a Dynamic Table in this skill?

A Stream tracks changes on a table for manual processing via Tasks, while a Dynamic Table declaratively refreshes its content based on a target lag time without requiring manual task scheduling.

Does this skill support Task DAGs?

Yes, it provides specific instructions for creating root and child tasks to orchestrate complex, multi-stage data processing workflows.

Can this skill help me troubleshoot failed Snowflake tasks?

Yes, it includes monitoring commands and error-handling guides for common issues like suspended tasks, stale streams, and merge conflicts.

How do I enable a task after creating it?

The skill includes the necessary SQL commands, such as 'ALTER TASK RESUME', to activate tasks after they are defined, as tasks are created in a suspended state by default.

Snowflake ELT & Data Transformation

Name: Snowflake ELT & Data Transformation
Author: jeremylongshore

byjeremylongshore

•

2,024

•

Gestión de Bases de Datos

Automates the creation and management of Snowflake data pipelines using streams, tasks, and dynamic tables.

This skill enables developers to build sophisticated ELT (Extract, Load, Transform) pipelines within the Snowflake ecosystem. It provides patterns for implementing Change Data Capture (CDC) via streams, scheduling transformations with tasks, orchestrating complex Task DAGs (Directed Acyclic Graphs), and utilizing declarative Dynamic Tables for automatic data freshness. It is ideal for data engineers and analysts looking to streamline their Snowflake data processing workflows with best practices for monitoring, error handling, and performance optimization.

Características Principales

01Change Data Capture (CDC) implementation using Snowflake Streams

02Orchestration of complex Task DAGs for multi-step processing

03Declarative data transformation with Dynamic Tables

04Automated task scheduling and transformation logic

05Built-in monitoring for task history and stream health

062,024 GitHub stars

Casos de Uso

01Implementing incremental data loads to minimize compute costs

02Managing complex data dependencies between multiple transformation layers

03Building real-time analytics pipelines for e-commerce order tracking

Características Principales

01Change Data Capture (CDC) implementation using Snowflake Streams

02Orchestration of complex Task DAGs for multi-step processing

03Declarative data transformation with Dynamic Tables

04Automated task scheduling and transformation logic

05Built-in monitoring for task history and stream health

062,024 GitHub stars

Casos de Uso

01Implementing incremental data loads to minimize compute costs

02Managing complex data dependencies between multiple transformation layers

03Building real-time analytics pipelines for e-commerce order tracking