Langfuse Annotation Manager

概要

The Langfuse Annotation Manager skill empowers developers to streamline their LLM evaluation workflows by managing human-in-the-loop annotations. It provides a comprehensive suite of tools to create, update, and delete trace scores across numeric, categorical, and boolean data types. By enabling users to quickly identify pending traces and export annotation data to JSON or CSV, it bridges the gap between observability and actionable quality improvements for AI applications.

主な機能

• Identify pending traces requiring human review based on timeframes
• 0 GitHub stars
• Create numeric, categorical, and boolean scores for traces
• Update or delete existing trace scores and comments
• Export annotation data to JSON and CSV formats for analysis
• View and list available score configurations and types

ユースケース

• Managing large-scale annotation workflows to find and review unrated traces
• Generating labeled datasets for fine-tuning by exporting verified scores
• Conducting manual QA and human-in-the-loop evaluation of LLM outputs