Serge

•
5.7k
•

Provides a self-hosted web interface and API for interacting with large language models via llama.cpp.

Serge is a robust, self-hosted web interface designed for seamless interaction with large language models (LLMs) such as Alpaca, leveraging the efficient llama.cpp library. It offers a fully dockerized environment, ensuring easy deployment and management, alongside a user-friendly API for programmatic access. The solution features a SvelteKit frontend for an intuitive chat experience, uses Redis for persistent chat history and parameter storage, and integrates FastAPI with LangChain for a powerful backend that wraps llama.cpp calls, making LLM access straightforward and private.

Key Features

01Web interface built with SvelteKit
02Persists chat history and parameters using Redis
03Self-hosted chat interface for LLMs
04Robust API powered by FastAPI and LangChain
05Fully Dockerized for easy deployment
065,724 GitHub stars

Use Cases

01Building custom applications that interact with local LLMs via an API
02Setting up a private, self-contained AI chat environment
03Running local LLMs without cloud API keys