LLM Fine-Tuning Knowledge Base

关于

This skill serves as a specialized domain-specific reference for developers working with Large Language Models, offering immediate access to structured data on model architectures, training methods, and optimization strategies. It provides actionable advice on configuring techniques like LoRA, QLoRA, and DPO, while offering comparative insights into popular models such as Qwen, DeepSeek, and Llama. By integrating deep technical knowledge directly into the workflow, it helps developers resolve training issues like overfitting or loss divergence and helps in selecting the most cost-effective models for specific tasks like reasoning or Chinese NLP.

主要功能

• Model selection guides for Qwen, DeepSeek, Llama, and Phi series
• Comparative analysis of alignment methods including DPO, ORPO, KTO, and SimPO
• Detailed configuration templates for LoRA, QLoRA, and DoRA optimization
• 0 GitHub stars
• Troubleshooting matrices for common training issues like overfitting and loss divergence
• Architecture deep-dives into Dense, Mixture of Experts (MoE), and MLA

使用场景

• Implementing preference optimization workflows to align models with human feedback
• Optimizing LoRA hyperparameters (rank, alpha) based on specific dataset sizes
• Selecting the most cost-effective and performant model for multilingual or reasoning-heavy tasks