What data formats are supported by these scripts?

The skill supports JSONL, CSV, Parquet, and Arrow formats for comprehensive dataset validation and quality checking.

Does it work with specific ML frameworks?

It is designed to support major ML frameworks including PyTorch, TensorFlow, and JAX, with specific checks for model weights and configs.

Can I use custom validation rules?

Yes, the skill allows you to provide custom JSON schemas to enforce specific data structures, required fields, and value ranges for your unique datasets.

Can it check GPU availability before training?

Yes, it includes specialized scripts to verify CUDA versions, GPU count, driver compatibility, and minimum VRAM requirements.

How does it help prevent training failures?

By performing 'dry runs' of the pipeline and validating token counts and data schemas, it catches common errors before they cause expensive mid-training crashes.

ML Training Validation Scripts

Name: ML Training Validation Scripts
Author: vanman2024

byvanman2024

0•

数据科学与机器学习

Ensures the integrity of machine learning training workflows by validating datasets, model checkpoints, and system dependencies.

This skill provides a comprehensive suite of production-ready validation and testing utilities designed specifically for machine learning projects. It helps developers maintain high data quality, verify model configuration integrity, and ensure end-to-end pipeline correctness before committing to expensive training runs. By automating checks for schema compliance, GPU availability, and model weight integrity, it significantly reduces the risk of training failures and silent data corruption in complex AI development workflows.

主要功能

01Model checkpoint and configuration verification for PyTorch and TensorFlow.

020 GitHub stars

03End-to-end pipeline testing with sample data and loss range checks.

04Detailed JSON reporting for validation passes and failures.

05Automated dataset validation with schema enforcement and quality metrics.

06System dependency and GPU/VRAM availability auditing.

使用场景

01Verifying large-scale training datasets for format compliance and label distribution before fine-tuning.

02Auditing remote server environments to ensure correct CUDA versions and sufficient VRAM exist.

03Debugging failed training runs by isolating issues in the data-loading or model-initialization phases.

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add vanman2024/ai-dev-marketplace validation-scripts

For use in Claude.ai and ChatGPT

主要功能

01Model checkpoint and configuration verification for PyTorch and TensorFlow.

020 GitHub stars

03End-to-end pipeline testing with sample data and loss range checks.

04Detailed JSON reporting for validation passes and failures.

05Automated dataset validation with schema enforcement and quality metrics.

06System dependency and GPU/VRAM availability auditing.

使用场景

01Verifying large-scale training datasets for format compliance and label distribution before fine-tuning.

02Auditing remote server environments to ensure correct CUDA versions and sufficient VRAM exist.

03Debugging failed training runs by isolating issues in the data-loading or model-initialization phases.

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add vanman2024/ai-dev-marketplace validation-scripts

For use in Claude.ai and ChatGPT