How can I make my training faster?

This skill recommends using Unsloth for 2x faster training and reduced memory consumption compared to standard HuggingFace trainers.

Does it cover dataset preparation?

Yes, it provides code for loading datasets from HuggingFace Hub, creating prompt templates, and tokenizing data correctly for LLMs.

Can I resume training from a checkpoint?

Absolutely. The skill includes standardized methods for saving checkpoints and resuming training using the HuggingFace Trainer API.

What frameworks are supported for fine-tuning?

The skill supports PyTorch, HuggingFace Transformers, Datasets, and the TRL library, with a strong recommendation for Unsloth for performance.

Does this skill handle instruction tuning?

Yes, it includes specific patterns for Supervised Fine-Tuning (SFT) using the SFTTrainer for chat and instruction-following tasks.

Model Fine-Tuning & Training

Name: Model Fine-Tuning & Training
Author: atrawog

byatrawog

0•

Data Science & ML

Provides expert guidance and implementation patterns for fine-tuning Large Language Models using PyTorch and HuggingFace.

This skill equips Claude with specialized knowledge for adapting pre-trained LLMs to specific tasks or domains using industry-standard frameworks like PyTorch and HuggingFace. It covers the entire training lifecycle, from dataset preparation and tokenization to advanced techniques like Supervised Fine-Tuning (SFT) and checkpoint management. By including recommendations for high-performance libraries like Unsloth, it helps developers optimize training speed and memory efficiency, making it an essential resource for building custom AI models and instruction-following agents.

Key Features

01Optimized training workflows using Unsloth for 2x faster performance

02End-to-end dataset preparation, tokenization, and formatting guides

03Manual PyTorch training loops for full customization and control

04Comprehensive HuggingFace Trainer and SFTTrainer implementation patterns

05Detailed hyperparameter guidance and checkpoint management strategies

060 GitHub stars

Use Cases

01Adapting a base LLM to follow specific domain-specific instructions or brand voices

02Improving model accuracy on niche datasets like legal, medical, or technical documentation

03Implementing efficient parameter-tuning workflows to reduce hardware requirements

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add atrawog/bazzite-ai-plugins finetuning

For use in Claude.ai and ChatGPT

Download Skill