How does this skill handle experiment versioning?

It implements an automated run-versioning system (e.g., @run-v0001) that archives previous iterations while maintaining the latest model in a primary directory.

Is the pipeline restricted to specific neural network architectures?

No, the pipeline is model-type agnostic. It uses a registry system to resolve ModelInstance classes based on your YAML configuration.

What is a ModelSet in the HAIPipe framework?

A ModelSet is a comprehensive asset package that bundles the trained model, its configuration, evaluation results, training logs, and metadata for full reproducibility.

Which remote storage providers are supported for model syncing?

The skill supports pushing and pulling ModelSet assets to and from S3, Google Cloud Storage (GCS), Databricks, and Google Drive.

Does it automatically handle feature engineering?

Yes, it creates and attaches a PreFnPipeline to the ModelSet during training to ensure the same feature transformations are applied during inference.

HAIPipe NN ModelSet & Pipeline Orchestrator

Name: HAIPipe NN ModelSet & Pipeline Orchestrator
Author: jluo41

byjluo41

0•

데이터 과학 및 ML

Orchestrates neural network training, evaluation, and deployment through a model-agnostic pipeline and standardized asset packaging system.

This skill provides the final orchestration layer (Layer 4) for the HAIPipe neural network framework, managing the complete lifecycle of machine learning models. It transforms trained instances into portable 'ModelSets' that bundle weights, training results, evaluation metrics, and lineage tracking into a single versioned asset. Designed for high-stakes research and production environments, it simplifies complex workflows through YAML-driven configurations and built-in support for remote synchronization with cloud providers like S3 and GCS.

주요 기능

01YAML-driven orchestration for model-agnostic training and inference flows.

02Standardized ModelSet packaging for training, evaluation, and lineage tracking.

030 GitHub stars

04Automated run versioning to track and archive multiple experiment iterations.

05Integrated feature pipeline attachment and automated example generation.

06Remote sync capabilities for S3, GCS, Databricks, and Google Drive storage.

사용 사례

01Automating multi-stage ML pipelines from data ingestion to metric reporting.

02Managing enterprise model registries across diverse cloud and local environments.

03Standardizing model packaging and versioning for reproducible academic research.

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add jluo41/research-skills haipipe-nn-4-modelset

For use in Claude.ai and ChatGPT

Download Skill