Ollama Proxy
Routes and load-balances LLM requests across multiple distributed Ollama instances through a single unified endpoint.
Ollama Proxy (Ollama Herd) aggregates multiple distributed Ollama instances across Mac, Linux, and Windows machines into a single drop-in replacement endpoint for localhost:11434. It intelligently load-balances inference, embeddings, image generation, and speech-to-text requests across available hardware using seven telemetry signals including thermal state, free memory, queue depth, and latency history. Designed for developers running self-hosted AI models, this skill provides zero-configuration node discovery, automated failover retries, and native compatibility with tools like Open WebUI, Aider, and LangChain without sending any data off your local network.