MTEB Leaderboard Navigator

About

This skill provides expert guidance for navigating the complex and rapidly evolving landscape of machine learning benchmarks, specifically the Massive Text Embedding Benchmark (MTEB). It helps users overcome common challenges such as locating historical data snapshots, identifying the most reliable data sources—ranging from GitHub repositories to API endpoints—and interpreting varying model naming conventions. By prioritizing temporal accuracy and cross-reference verification, it ensures that model comparisons and performance data are precise, relevant, and properly contextualized for specific dates or technical requirements.

Key Features

• Format interpretation for diverse model naming conventions and API formats
• Cross-source verification and validation protocols to ensure data integrity
• Guidance on avoiding outdated academic paper snapshots in favor of live data
• Temporal accuracy strategies for historical and current data retrieval
• Identification of primary data sources including GitHub repositories and API endpoints
• 16 GitHub stars

Use Cases

• Finding the top-performing embedding models for a specific language as of a specific historical date
• Verifying current MTEB rankings for specific metrics or subsets when web interfaces are inaccessible
• Extracting raw leaderboard data from GitHub commit histories to track model performance trends