vMLX
Run large language models, vision models, and image generation locally on Apple Silicon Macs with an OpenAI and Anthropic compatible API.
vMLX is a powerful local AI engine designed specifically for Apple Silicon Macs, enabling users to run a wide array of large language models (LLMs), vision-language models (VLMs), and image generation models entirely on their machine. It offers an OpenAI and Anthropic compatible API, ensuring seamless integration with existing AI workflows without the need for cloud services, API keys, or data leaving the device. With advanced features like continuous batching, paged KV cache, and JANG 2-bit KV cache quantization, vMLX optimizes performance and memory usage, making local AI development both efficient and private.