Multi-LLM Hub

This Multi-LLM Hub serves as a production-grade, secure starting point for AI projects, addressing prevalent security flaws like hardcoded keys and lack of sandboxing in the AI ecosystem. It implements a Guardian pattern where secrets are isolated and injected, ensuring structural sandboxing where application modules cannot access raw environment variables. Built on an async Quart + Hypercorn stack for efficient Server-Sent Events (SSE) handling, it orchestrates various LLM and search providers, offering graceful degradation, an LLM fallback chain, and a unique two-database architecture for distinct security and performance responsibilities.

Key Features

Use Cases