How does this differ from LangSmith?

Langfuse is an open-source, often self-hosted alternative to LangSmith that provides similar tracing and evaluation capabilities with built-in prompt management and a focus on production control.

Does it support LangChain or LangGraph?

Absolutely. It includes specific patterns for integrating the Langfuse CallbackHandler directly into LangChain and LangGraph workflows for automatic instrumentation.

How does prompt management work in this skill?

It utilizes the Langfuse prompt registry, enabling you to manage prompt versions and environment labels (e.g., 'production') to update AI behavior without requiring a code redeploy.

Can I track costs for specific users or sessions?

Yes, the skill includes implementation patterns for session and user tracking, allowing you to attribute token usage and costs to specific user IDs in your dashboard.

Can I use LLMs to evaluate other LLM responses?

Yes, this skill provides patterns for 'Multi-Judge' evaluation and G-Eval, which use secondary LLMs to automatically score the quality of your primary model's output.

Langfuse LLM Observability

Name: Langfuse LLM Observability
Author: yonatangross

byyonatangross

•

분석 및 모니터링

Implements comprehensive LLM observability, tracing, and prompt management using the open-source Langfuse platform.

This skill enables developers to integrate Langfuse into their LLM applications for robust production monitoring and performance optimization. It provides standardized patterns for distributed tracing using the @observe decorator, automatic token and cost tracking, and a centralized prompt management system with native versioning. Whether you are debugging complex multi-agent workflows, migrating from LangSmith, or implementing automated G-Eval quality assessments, this skill offers the production-ready infrastructure needed to maintain visibility and control over LLM performance, latency, and expenses.

주요 기능

01Distributed tracing for multi-step agentic workflows and nested spans

02Version-controlled prompt management with A/B testing and labels

03Real-time token usage and cost tracking with custom model pricing

04Automated LLM output evaluation using G-Eval and multi-judge patterns

05Deep integration with LangChain and LangGraph via CallbackHandlers

0629 GitHub stars

사용 사례

01Managing and testing prompt templates without hardcoding in the backend

02Setting up production-grade LLM monitoring and cost dashboards

03Debugging and optimizing complex, multi-turn AI conversations

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add yonatangross/skillforge-claude-plugin langfuse-observability

For use in Claude.ai and ChatGPT

Download Skill