Can I use this for DPO or GRPO datasets?

Yes, the synthesizer includes specific interactive flows to configure pipelines for SFT, DPO, and GRPO dataset creation.

Does it support local LLMs like Ollama?

Yes, it can configure pipelines to use any OpenAI-compatible API, including local deployments like Ollama, vLLM, and LM Studio.

What input data sources are supported?

Pipelines can be configured to use local files (CSV, Parquet, JSONL), HuggingFace datasets, SQL databases via connectorx, or generated from scratch.

How does it handle data quality and deduplication?

The skill can generate pipeline steps for exact hash deduplication, fuzzy simhash deduplication, semantic embedding-based filtering, and JSON schema validation.

TweakTune is a Rust-powered, Python-facing library designed for building scalable pipelines to synthesize training data for Large Language Models.

TweakTune Data Synthesizer

Name: TweakTune Data Synthesizer
Author: qooba

byqooba

•

데이터 과학 및 ML

Designs and generates high-performance pipelines for synthesizing high-quality LLM training datasets, conversations, and structured data.

TweakTune Data Synthesizer is an interactive assistant that streamlines the creation of synthetic training data for large language models. By guiding users through a structured discovery process, it generates production-ready Python code for the TweakTune library to create datasets for Supervised Fine-Tuning (SFT), DPO, and GRPO. It automates complex tasks like multi-turn conversation building, function-calling simulation, and data validation, leveraging Rust-powered performance for efficient, scalable data synthesis.

주요 기능

01Advanced deduplication via hash, simhash, and embeddings

022 GitHub stars

03Multi-provider LLM support including OpenAI, Azure, and local APIs

04Automated data validation for JSON schemas and conversation formats

05Interactive Q&A for rapid pipeline prototyping

06Multi-turn conversation and function-calling dataset generation

사용 사례

01Generating synthetic datasets for LLM fine-tuning (SFT/DPO/GRPO)

02Creating structured JSON datasets from unstructured text or seeds

03Simulating complex tool-use and function-calling dialogues for agent training

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add qooba/tweaktune tweaktune-synthesizer

For use in Claude.ai and ChatGPT

주요 기능

01Advanced deduplication via hash, simhash, and embeddings

022 GitHub stars

03Multi-provider LLM support including OpenAI, Azure, and local APIs

04Automated data validation for JSON schemas and conversation formats

05Interactive Q&A for rapid pipeline prototyping

06Multi-turn conversation and function-calling dataset generation

사용 사례

01Generating synthetic datasets for LLM fine-tuning (SFT/DPO/GRPO)

02Creating structured JSON datasets from unstructured text or seeds

03Simulating complex tool-use and function-calling dialogues for agent training

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add qooba/tweaktune tweaktune-synthesizer

For use in Claude.ai and ChatGPT