Evals

Evals aren't a step at the end. They run the whole way through
Evals aren't a step at the end. They run the whole way through

There’s a version of building an AI app that goes like this. You build the thing, you get it …

Four ways to run an eval, from a cheap unit test to a full-blown agent
Four ways to run an eval, from a cheap unit test to a full-blown agent

Someone asked me last week how you actually run an eval on an AI app. I gave the honest answer, …

How to build an eval you can actually trust
How to build an eval you can actually trust

Here’s how most people build an eval. They open a file, write an LLM judge prompt that says …