What are the typical costs associated with running this pipeline?

The total estimated cost is approximately $2.00, which includes roughly $0.50 for instruction generation using fast LLMs and $1.50 for 15 minutes of training on Tinker.

How does it prevent the model from just memorizing the book?

It uses 15+ different prompt templates and 5+ system prompts to ensure the model learns the underlying prose style and rhythm rather than just the literal plot content.

Can I use this for instruction-tuned models?

While possible, the skill recommends using base models (like Qwen-8B-Base) because they are more malleable and capture low-level stylistic patterns more effectively than pre-instructed versions.

Why does this skill prefer ePub over PDF files?

ePub files allow for clean paragraph-level extraction and avoid the OCR errors common in PDFs, which can otherwise be learned as bad patterns by the AI model during the training process.

What is the Book SFT Pipeline?

It is a specialized capability for Claude Code that automates the conversion of books into training datasets for fine-tuning Large Language Models on specific writing styles.

Book SFT Pipeline

Name: Book SFT Pipeline
Author: muratcankoylan

bymuratcankoylan

•

7,304

•

数据科学与机器学习

Automates the end-to-end process of converting literary works into high-quality datasets for fine-tuning AI models on specific authorial voices and writing styles.

The Book SFT Pipeline provides a comprehensive framework for transforming raw ePub files into specialized Supervised Fine-Tuning (SFT) datasets. By employing an orchestrator-based architecture, it manages semantic text segmentation, diverse instruction generation, and LoRA training configurations. This skill is ideal for developers and researchers building style-transfer models or looking to replicate distinctive prose styles in LLMs like Qwen or Llama, ensuring high fidelity to the original author's rhythm and vocabulary while preventing over-memorization through multi-variant instruction templates.

主要功能

01Comprehensive validation suite including modern scenario testing and originality verification

02Semantic segmentation using word-count boundaries and overlaps to maintain context

03Automated JSONL dataset construction optimized for LoRA training on platforms like Tinker

04Multi-agent instruction generation utilizing 15+ diverse templates to prevent overfitting

05Intelligent ePub text extraction that preserves paragraph structures and filters meta-matter

067,304 GitHub stars

使用场景

01Creating a LoRA adapter that writes in the style of a specific classic or modern author

02Training 8B-parameter base models to adopt distinctive professional or creative voices

03Building high-quality synthetic instruction datasets from long-form literary content

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add muratcankoylan/agent-skills-for-context-engineering book-sft-pipeline

For use in Claude.ai and ChatGPT

Download Skill