Evaluates LLM outputs and optimizes prompts using Evidently.ai metrics and LLM-as-a-judge patterns.
This skill integrates Evidently.ai into the Claude Code workflow to provide a robust framework for assessing and improving LLM performance. It enables developers to implement automated quality checks through text descriptors, set up sophisticated LLM-as-a-judge evaluators for qualitative metrics, and perform automated prompt tuning. Whether you're measuring RAG accuracy, comparing model variations, or monitoring production quality, this skill provides the necessary tools to transition from subjective assessment to data-driven LLM development within your Jupyter environment.
主な機能
01Automated prompt optimization for classification and generation tasks
02LLM-as-a-Judge patterns for automated qualitative assessment
03Row-level text descriptors for sentiment, length, and validity
04Comprehensive HTML and JSON reporting for performance monitoring
05RAG quality evaluation focusing on relevance and factuality
060 GitHub stars
ユースケース
01Evaluating RAG pipeline output for hallucination and relevance
02Optimizing system prompts for higher classification accuracy
03Benchmarking performance differences between model versions
