Minions
Enables collaboration between small, on-device LLMs and larger cloud-based models for cost-efficient language processing.
概要
Minions is a communication protocol that empowers small, on-device language models to work in concert with powerful cloud-based LLMs. By primarily utilizing local models for context reading, it drastically reduces cloud costs while maintaining, or even improving, output quality. This approach is particularly beneficial for tasks requiring extensive contextual understanding without the expense of sending large amounts of data to the cloud.
主な機能
- Facilitates collaboration between local and remote LLMs
- Supports Ollama and Tokasaurus for local model serving
- Includes demonstration application and example code
- Reduces cloud costs by processing context locally
- Compatible with OpenAI, TogetherAI, DeepSeek and other cloud LLM providers.
- 529 GitHub stars
ユースケース
- Privacy-preserving data analysis with LLMs
- Cost-effective question answering over large documents
- Accelerated LLM processing with distributed workloads
Sponsored