vMLX

•
924
•

Run large language models, vision models, and image generation locally on Apple Silicon Macs with an OpenAI and Anthropic compatible API.

vMLX is a powerful local AI engine designed specifically for Apple Silicon Macs, enabling users to run a wide array of large language models (LLMs), vision-language models (VLMs), and image generation models entirely on their machine. It offers an OpenAI and Anthropic compatible API, ensuring seamless integration with existing AI workflows without the need for cloud services, API keys, or data leaving the device. With advanced features like continuous batching, paged KV cache, and JANG 2-bit KV cache quantization, vMLX optimizes performance and memory usage, making local AI development both efficient and private.

Características Principales

01High-performance local inference with continuous batching and speculative decoding
02Advanced KV cache optimizations including paged, prefix, and disk caching with quantization
03OpenAI and Anthropic compatible API for LLMs, VLMs, embeddings, and audio models
04Local image generation and editing via Flux models with dedicated API endpoints
05Built-in support for LLM tool calling and reasoning/thinking modes
0610 GitHub stars

Casos de Uso

01Developing AI applications that leverage LLMs, VLMs, and image generation locally on macOS
02Integrating private, on-device AI capabilities into existing OpenAI or Anthropic SDK workflows
03Experimenting with and deploying various MLX-compatible models for chat, agentic functions, and content creation