LLM Streaming & Real-time Patterns
Implements production-ready real-time token streaming, SSE endpoints, and tool-call handling for AI applications.
This skill provides standardized implementation patterns for delivering LLM responses in real-time, significantly improving user experience by reducing time-to-first-token. It covers the full lifecycle of a streaming response, from basic OpenAI async generators and FastAPI Server-Sent Events (SSE) to complex frontend consumption and tool-call accumulation. By incorporating best practices for backpressure management and error handling, it ensures your AI-driven interfaces remain responsive and robust even during high-latency model generations or complex tool-use scenarios.
主な機能
01Asynchronous token streaming patterns
0269 GitHub stars
03FastAPI Server-Sent Events (SSE) implementation
04Streaming tool-call accumulation and execution
05Frontend stream consumption and UI update logic
06Backpressure and buffer management for slow consumers
ユースケース
01Implementing real-time progress updates for long-running AI agents
02Building low-latency AI chat interfaces and dashboards
03Managing high-concurrency streaming endpoints in production