LB-10 Eval Lab

Measures the prompts behind every other system, so each prompt change ships with a score.

LB-10

Eval Lab Evaluation board

Live

Eval Lab measures the prompts behind the other systems. Pick one, edit it, and run your version and production's on the same ten cases of that system's golden set, on one or two free providers. The report compares how many cases each passed, with how sure ten cases can make that, the latency, the tokens and the calls, and opens every case that changed.

A visitor's run is graded by rules, never by a model: each reply is checked as it was written and never repaired. Production's results are cached for everybody, so most runs make about ten calls a provider.

Start from a prepared edit

Choose a prepared editLB-01 drafter: a word limitOne rule added: keep the reply under 80 words.LB-01 classifier without its JSON ruleThe line that asks for one JSON object is replaced by headings.LB-02 planner records details laterIt is told to record the details only once it has them all.LB-05 SQL writer, unchangedProduction's prompt as it is, scored on its own.LB-08 generator without its exampleThe worked example is taken out of the prompt.

The edit

Adds this line

- Keep the whole reply under 80 words.

after this one

- If the sources don't answer the ticket, set "answerable" to false and leave "sentences" empty: a person will take over.

Runs onGroq

A live run uses your one run of the day.

1. The prompt to measure

The Scope

Start a run and its trace appears here, step by step.

Operating limit	Value
Cases per visitor run	10
Runs per visitor per day	1
Result cache	By prompt hash
Model calls per visitor run (est.)	about 20