Llm
Evals do three jobs, not one
Ask most people what an eval is for and you’ll get some version of “testing”. You …
Anatomy of an evaluator: what happens when your prompt meets your traces
Last time I pulled apart the eval prompt - the role, criteria, rubric and examples you write to tell …
Anatomy of an eval prompt: what to actually put in it
When people decide to use an LLM as a judge, the prompt they reach for first is almost always some …
Spans, traces and sessions: the three zoom levels of an AI app
The moment you start tracing an AI app, three words turn up everywhere: span, trace, session. People …
Evals aren't a step at the end. They run the whole way through
There’s a version of building an AI app that goes like this. You build the thing, you get it …
Four ways to run an eval, from a cheap unit test to a full-blown agent
Someone asked me last week how you actually run an eval on an AI app. I gave the honest answer, …
How to build an eval you can actually trust
Here’s how most people build an eval. They open a file, write an LLM judge prompt that says …
Build a ChatGPT clone in .NET using OpenAI and Semantic Kernel
ChatGPT is the latest big thing, reaching over 100 million users in just 2 months. It’s being …