Scorable FAQs

Question 1

What are Root Signals evaluators and judges in the context of Scorable?

Accepted Answer

Root Signals evaluators are the underlying quality criteria models that Scorable exposes as tools. Judges are collections of these evaluators, forming a powerful 'LLM-as-a-judge' system capable of comprehensive, multi-faceted assessment of AI responses.

Question 2

How does Scorable help improve AI agent performance?

Accepted Answer

Scorable empowers AI agents to automatically assess the quality of their LLM responses, adhere to coding policies, and leverage feedback from 'LLM-as-a-judge' evaluators. This continuous evaluation loop allows agents to iteratively refine and enhance their outputs for superior performance.

Question 3

What types of evaluations can Scorable perform?

Accepted Answer

Scorable supports various evaluation types, including running standard evaluations by evaluator ID or name, performing coding policy adherence checks using policy documents, and executing judge-based evaluations, which leverage a collection of Root Signals evaluators as an LLM-as-a-judge system.

Question 4

Is Scorable compatible with other AI development tools?

Accepted Answer

Yes, Scorable is built with broad compatibility in mind. It implements Server-Sent Events (SSE) for network deployment and is compatible with various Model Context Protocol (MCP) clients, such as Cursor, making integration into existing AI development workflows seamless.

Question 5

What is Scorable and its primary purpose?

Accepted Answer

Scorable is a Model Context Protocol (MCP) server that connects AI assistants and agents to Root Signals evaluators. Its primary purpose is to enable these AI entities to evaluate responses against diverse quality criteria, acting as a measurement and control system for LLM automations.

Scorable

About

Key Features

Use Cases