🧰Custom Scorer

The Custom Scorer page on the Evaluable AI platform allows you to:

• View pre-defined LLM and heuristic scorers

• Create your own LLM and heuristic scorers, either by cloning a pre-defined scorer or building one from scratch!

Page Components

There are two tabs that can be toggled between on the top left of the page — LLM and Heuristic.

LLM: Using LLMs as scorers for evaluating other LLMs allows for organizations to enhance the way they evaluate, develop, and deploy language models. To learn about how to leverage and create custom heuristic scorers, navigate to the LLM-based Scorers page.

Heuristic: Heuristic evaluations compare the output from models against a gold standard or the best possible output. To learn about how to leverage and create custom heuristic scorers, navigate to the Heuristic Scorers page.