H2oGPT
Engage in private, local conversations with large language models, supporting various document types including images, videos, and PDFs, ensuring 100% data privacy.
About
H2oGPT is an Apache V2 licensed open-source project that allows users to query and summarize their documents and chat with local, private GPT LLMs. It supports an extensive range of document types, including PDFs, Excel, Word, images, video frames, YouTube, audio, code, and text, maintaining a persistent and private offline database. The tool is designed for efficient context use with instruct-tuned LLMs and offers parallel summarization and extraction for fast performance. It supports a wide variety of models like LLaMa2, Mistral, and Falcon, with comprehensive GPU and CPU compatibility, and provides a user-friendly Gradio UI or CLI for interaction.
Key Features
- OpenAI compliant API server acting as a drop-in replacement for chat, text completions, audio, image generation, and embeddings.
- Supports a wide variety of LLM models (LLaMa2, Mistral, Falcon, Vicuna, WizardLM) with GPU, CPU, and attention sinks for long generation.
- Intuitive Gradio UI or CLI with streaming, document upload, vision models (LLaVa, Claude-3), image generation (Stable Diffusion), and voice STT/TTS.
- Private offline database for diverse document types (PDFs, Excel, Word, Images, Video, Audio, Code, Text) with persistent storage.
- 11,833 GitHub stars
- Integration with Web-Search, function tool calling, JSON mode, and agents for search, document Q/A, Python Code, and CSV frames.
Use Cases
- Developing and testing AI applications with an OpenAI-compatible local server for chat, embeddings, and multimedia generation.
- Efficiently querying, summarizing, and extracting information from personal or enterprise documents offline.
- Conducting private and secure local chats with large language models without sending data to external services.
