Evals
Anatomy of an eval prompt: what to actually put in it
When people decide to use an LLM as a judge, the prompt they reach for first is almost always some …
Spans, traces and sessions: the three zoom levels of an AI app
The moment you start tracing an AI app, three words turn up everywhere: span, trace, session. People …
Evals aren't a step at the end. They run the whole way through
There’s a version of building an AI app that goes like this. You build the thing, you get it …
Four ways to run an eval, from a cheap unit test to a full-blown agent
Someone asked me last week how you actually run an eval on an AI app. I gave the honest answer, …
How to build an eval you can actually trust
Here’s how most people build an eval. They open a file, write an LLM judge prompt that says …