Field notes from the eval lab.

Accuracy on a static set tells you little about an agent that calls tools. Here is what we measure instead — and why.

Fewer obligations than the headlines suggest, more evidence than most teams keep. A practical reading.

One trace, eleven spans and a retrieval step that returned the wrong policy. How we found it in four minutes.

An automated grader is only useful if it agrees with your experts. Here is how we measure that agreement.

Policy checks that add no noticeable latency: small models, early exits and a lot of caching.

Mirror real requests to a new agent, score everything, change nothing. The safest way to find out what a release will do.