~/
kellerstein
writing
about
rss
Writing
1 post · tagged evaluation · oldest Mar 2025
all
agents
17
architecture
11
infra
10
ai
8
kubernetes
7
azure
4
debugging
4
python
4
testing
4
docker
2
observability
2
product
2
qemu
2
refactoring
2
tooling
2
windows
2
astro
1
automation
1
evaluation
1
fastapi
1
git
1
helm
1
mcp
1
meta
1
mitmproxy
1
models
1
retrospective
1
reverse-engineering
1
security
1
terraform
1
traefik
1
OpenOperator
1 post · 2025
watch demo
2025·03·26
"It looked like it worked": grading a non-deterministic agent
The least trustworthy signal in the system is the agent reporting success. Here is the evaluator we built instead — and the four-word commit that forced a whole second repository into existence.
agents
testing
evaluation
7 min