How is data integrity verified after migration?

The skill includes a validation module that compares source and target systems for row counts, schema consistency, and sample data matches.

How does it handle complex schema mapping?

It includes a dedicated schema migration script that maps problematic legacy types (like char or specific decimals) to Delta Lake-compatible Spark types automatically.

Does it support incremental data loads?

Yes, the data migration patterns include logic for batch processing and can be configured with incremental columns to handle large-scale transfers.

Which source systems are supported by this migration skill?

The skill provides patterns for migrating from on-premises Hadoop (Hive), Snowflake, AWS Redshift, Azure Synapse, and legacy RDBMS like Oracle or Teradata.

Databricks Migration Deep Dive

Name: Databricks Migration Deep Dive
Author: jeremylongshore

byjeremylongshore

•

1,243

データベース管理

Orchestrates comprehensive platform migrations from legacy data systems like Hadoop and Snowflake into Databricks using automated assessment and Delta Lake conversion patterns.

概要

This skill provides a robust framework for managing complex data migrations to Databricks. It guides users through the entire migration lifecycle, including automated discovery of legacy Hive metastores, intelligent schema mapping for Delta Lake compatibility, and high-performance data transfer scripts. Whether you are moving from on-premises Hadoop clusters or cloud warehouses like Redshift and Snowflake, this skill provides the implementation patterns, batch-processing logic, and validation protocols necessary to ensure a reliable transition to the Databricks Lakehouse architecture.

主な機能

Automated source system assessment and migration prioritization logic
Intelligent schema conversion from Hive/legacy SQL to Delta Lake types
Batch-based data migration scripts with checkpointing and partitioning
Tailored migration patterns for Hadoop, Snowflake, Redshift, and Synapse
Automated validation tools for row counts, schema matching, and data integrity
1,243 GitHub stars

ユースケース

Consolidating legacy data warehouses like Oracle or Teradata into Databricks
Migrating data workloads and schemas from Snowflake to Delta Lake
Modernizing on-premises Hadoop/Hive clusters to a Databricks cloud environment

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add jeremylongshore/claude-code-plugins-plus-skills databricks-migration-deep-dive

For use in Claude.ai and ChatGPT

Download Skill

GitHub

概要