LLM Response Streaming

概要

Enables the construction of high-performance, real-time chat interfaces by providing a standardized way to stream incremental LLM responses. This skill abstracts the complexities of different provider SDKs—including OpenAI, Anthropic, Google, and Ollama—into a single async iterator pattern. It simplifies handling chunk deltas, monitoring token usage, and detecting stream completion reasons, making it an essential tool for developers building responsive AI applications that require immediate feedback and low perceived latency.

主な機能

• Standardized StreamChunk schema for consistent data handling across models
• 3 GitHub stars
• Unified async iterator for cross-provider streaming (OpenAI, Anthropic, Gemini, Ollama)
• Real-time incremental construction of tool calls
• Automatic extraction of token usage statistics from the final stream chunk
• Built-in detection of finish reasons like stop, length, and tool_calls

ユースケース

• Displaying progressive content generation to improve user experience
• Real-time processing of text streams for live analysis or filtering
• Creating interactive AI chatbots with real-time UI updates