How does this skill prevent data leakage?

It implements point-in-time lookups using the Databricks FeatureStoreClient, ensuring training sets only contain data available at the specific timestamp of the event.

Does it support Databricks Unity Catalog?

Yes, the provided patterns are designed to work with catalog and schema structures compatible with Unity Catalog governance.

Can I use this for real-time model serving?

Yes, the skill includes patterns for publishing feature tables to online stores like CosmosDB or DynamoDB for sub-second feature retrieval during inference.

Is feature versioning handled automatically?

The skill provides patterns for versioning feature definitions and monitoring freshness to maintain a reliable lineage of your machine learning data.

What libraries does this skill utilize?

It primarily uses the databricks-feature-store client library along with PySpark for data transformations and aggregations.

Databricks Feature Engineering

Name: Databricks Feature Engineering
Author: vivekgana

byvivekgana

•

Data Science & ML

Streamlines the creation, management, and serving of scalable feature stores within the Databricks MLOps ecosystem.

About

This skill empowers Claude to architect production-grade feature engineering pipelines using the Databricks Feature Store. It provides standardized patterns for creating versioned feature tables, generating training datasets with point-in-time lookups to prevent data leakage, and publishing features to online stores for low-latency inference. By integrating best practices for metadata documentation and feature freshness, this skill bridges the gap between raw data engineering and high-performance machine learning model deployment.

Key Features

Standardized feature versioning and documentation patterns
Online feature serving configuration for real-time inference
Integration with PySpark for scalable feature transformations
Automated feature table creation and registration
Point-in-time correct training set generation
2 GitHub stars

Use Cases

Building reusable feature repositories across multiple ML projects
Eliminating data leakage during model training with feature lookups
Deploying real-time features to online databases for low-latency applications

About

Key Features

Standardized feature versioning and documentation patterns
Online feature serving configuration for real-time inference
Integration with PySpark for scalable feature transformations
Automated feature table creation and registration
Point-in-time correct training set generation
2 GitHub stars

Use Cases

Building reusable feature repositories across multiple ML projects
Eliminating data leakage during model training with feature lookups
Deploying real-time features to online databases for low-latency applications