How does this skill differ from LangSmith?

Langfuse is an open-source, self-hostable alternative to LangSmith. This skill provides patterns for tracing and evaluation that work with Langfuse's free and open-source ecosystem while offering built-in prompt management.

Does it support prompt versioning?

Absolutely. It leverages the Langfuse Prompt Registry, allowing you to manage prompt versions, use Jinja2 templates, and toggle between production and staging labels without code changes.

How do I track costs for different LLM providers?

The skill implements patterns that automatically capture token usage and map them to model-specific pricing, providing a centralized dashboard for monitoring your AI spend across various providers.

Can I use this with LangChain or LangGraph?

Yes, the skill includes integration patterns for using Langfuse CallbackHandlers, which allow for automatic instrumentation of LangChain and LangGraph workflows.

What is G-Eval and how is it used here?

G-Eval is a framework for using LLMs to evaluate other LLMs. This skill provides the infrastructure to set up automated quality scores for your traces based on custom criteria.

Langfuse AI Observability

Name: Langfuse AI Observability
Author: yonatangross

byyonatangross

•

Analíticas y Monitorización

Implements comprehensive LLM observability using Langfuse to trace calls, manage prompts, and track token costs in production AI applications.

Langfuse AI Observability is a specialized Claude Code skill designed to integrate the open-source Langfuse platform into your AI development workflow. It provides production-ready patterns for distributed tracing, version-controlled prompt management, and automated LLM evaluations. By utilizing the @observe decorator and structured session tracking, this skill enables developers to debug complex multi-agent spans, monitor real-time token expenses, and perform systematic A/B testing on prompt variants to ensure high-quality, cost-effective model performance.

Características Principales

01Session and user tracking for multi-turn conversation analytics

0269 GitHub stars

03Native prompt management with versioning, staging labels, and A/B testing

04Automated LLM evaluation using G-Eval and multi-judge scoring frameworks

05Distributed tracing with parent-child spans using the @observe decorator

06Real-time token usage and cost tracking for AI budget management

Casos de Uso

01Debugging slow or hallucinated LLM responses in complex agentic workflows

02Managing and versioning prompts in production without hardcoding logic

03Migrating from proprietary observability tools like LangSmith to self-hosted Langfuse

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add yonatangross/orchestkit langfuse-observability

For use in Claude.ai and ChatGPT

Download Skill