How do you know it actually works?
An AI product’s quality isn’t a feeling; it’s a measurement — and the measurement is a machine that can break, drift and lie. This page is ten mental models for building evals you can trust, one picture of how they fit together, and lessons from rebuilding a real eval system. Read it top to bottom; every idea opens in place.
The whole picture
Evals are a loop, not a test suite you write once: find what goes wrong, grade it honestly, fix each cause where it lives, prove the fix — then automate, and go round again. Click a step to jump to it below; the M and L chips are exactly the mental models and real-build lessons you’ll find under that step. Prefer prose? Read the whole loop as one note.
Walk the loop
The picture, step by step. Under each step: the mental model behind it (its core idea shown, go deeper opens the full model in place), then the lessons from rebuilding the Tide money coach’s eval system (September–October 2026) where it was learned the hard way.