Evaluations turn opinions about an AI system into measurements. A set of representative inputs is paired with expected outputs or scoring criteria, and the system is run against it. Metrics might include accuracy of extracted fields, correct classification, adherence to policy, cost, and latency.
Evals should be built before launch and run whenever prompts, models, data, or tools change. They catch regressions that a quick demo would miss and make it possible to compare models objectively. Including tricky and adversarial cases is as important as including typical ones.