Does this skill help with database setup?

Yes, it includes scripts and SQL templates to automatically set up the necessary tables and Row Level Security (RLS) policies in Supabase.

What is the Eval Tracking skill used for?

It is used to store and organize the results of LLM evaluations in a Supabase database, making it easy to track performance over time.

What metrics can I track with this skill?

The skill is flexible and allows you to track metadata for runs, specific test cases, and custom scores or metrics relevant to your specific AI application.

Does it provide queries for dashboards?

Yes, it includes a collection of pre-defined SQL queries designed to pull data for performance dashboards and analytics.

Can I use this to find regressions in my AI model?

Absolutely. By tracking scores across different evaluation runs, you can compare current results with historical data to identify any drops in quality.

LLM Evaluation Tracking

Name: LLM Evaluation Tracking
Author: vanman2024

byvanman2024

•

分析与监控

Manages and persists LLM evaluation metrics using Supabase to track model performance, regressions, and run history.

关于

The Eval Tracking skill provides a robust framework for developers to record, store, and analyze the performance of their AI models and prompts. By leveraging a Supabase backend, it automates the creation of structured tables for evaluation runs, individual test cases, and specific metric scores. This skill is essential for teams looking to build data-driven AI workflows, allowing for historical regression tracking and the generation of visual dashboards to monitor model quality over time.

主要功能

Pre-built SQL templates for dashboarding and analytics
Automated Supabase schema deployment for evaluation data
Structured tracking of evaluation runs, cases, and scores
2 GitHub stars
Support for regression testing and historical comparison
Built-in migration scripts for rapid database setup

使用场景

Creating centralized quality assurance dashboards for AI features
Detecting performance regressions in production AI workflows
Monitoring LLM performance across different model versions or prompt iterations

关于

主要功能

Pre-built SQL templates for dashboarding and analytics
Automated Supabase schema deployment for evaluation data
Structured tracking of evaluation runs, cases, and scores
2 GitHub stars
Support for regression testing and historical comparison
Built-in migration scripts for rapid database setup

使用场景

Creating centralized quality assurance dashboards for AI features
Detecting performance regressions in production AI workflows
Monitoring LLM performance across different model versions or prompt iterations