Ollaya

•
1.5k
•

Run open decision models locally to provide fast, calibrated probabilities for routing, guardrails, and agent decisions without generating text.

Ollaya is a specialized runtime and inference server that runs open decision models on local hardware, functioning for decision layers much like Ollama does for large language models. Rather than generating text tokens, it evaluates typed questions against structured inputs to return calibrated probabilities for routing, triage, tool approvals, and safety guardrails in milliseconds. Ollaya features full compatibility with TypeSafe's Jev API (/v1/systemone) and includes native Model Context Protocol (MCP) server capabilities, allowing autonomous agents in tools like Claude Code, Cursor, and custom pipelines to make rapid, private, on-device decisions.

主要功能

01Enables custom Modelfile creation and dataset-based temperature calibration
02Provides wire-level TypeSafe Jev compatibility via /v1/systemone, /v1/decisions, and /v1/models endpoints
031,473 GitHub stars
04Serves 29+ open decision model families locally with low latency and author-aligned calibration
05Integrates seamlessly with AI workflows via a built-in Model Context Protocol (MCP) server
06Accelerates inference with ONNX Runtime and llama.cpp across CUDA, ROCm, Metal, Vulkan, and CPU

使用案例

01Private Local Inference: Replacing cloud-hosted decision APIs like TypeSafe Jev with self-hosted models for privacy and compliance.
02Agent Routing and Tool Approval: Evaluating state context to approve sensitive tool execution or route requests without heavyweight LLM generations.
03Fast Triage and Moderation: Running automated ticket categorization, urgency scoring, and safety guardrails on incoming text or data.