Calista Research LLMs
CreatedShiroKatsuya
Enables on-device and cloud language models to collaborate, reducing cloud costs while maintaining quality by reading long contexts locally.
About
Calista Research LLMs (formerly Minions) facilitates cost-efficient collaboration between on-device and cloud language models. It employs a communication protocol where small, local models work with powerful cloud-based models. By enabling local models to read long contexts, the tool minimizes the need for cloud processing, thereby reducing cloud costs. The repository includes a demonstration of the protocol and examples of how to set up and use the system with various local and remote LLM clients.
Key Features
- Reduces cloud costs by processing long contexts locally
- Offers streaming output with custom callback functions
- Supports OpenAI and TogetherAI API keys for cloud models
- Compatible with Ollama and Tokasaurus for running local models
- Supports collaboration between on-device and cloud LLMs
- 0 GitHub stars
Use Cases
- Combining the speed of local models with the power of cloud models
- Cost-effective LLM processing for long documents
- Implementing LLM workflows in resource-constrained environments