Automates the evaluation, trace analysis, and documentation cycle for Langfuse-integrated AI agents.
The Langfuse Agent Evaluator skill enables developers to systematically optimize AI agents by orchestrating comprehensive evaluation cycles directly within Claude Code. It automates the execution of Langfuse experiments, performs deep-dive root cause analysis on failure patterns through trace comparison, and generates detailed remediation reports. By identifying specific failure dimensions and symptoms, this skill provides actionable recommendations for prompt or code improvements, ensuring that agent performance is driven by empirical data and structured observability.
Key Features
01Root cause analysis using comparative trace analysis
021 GitHub stars
03Dual-mode reporting via Linear issues or local Markdown documentation
04Actionable fix recommendations for prompts and logic
05Failure grouping by dimension and symptom patterns
06Automated Langfuse experiment and dataset execution
Use Cases
01Systematically identifying and debugging edge cases in complex agent workflows
02Generating structured technical debt reports for LLM performance improvements
03Regression testing AI agents after major prompt or model updates
