Writing
17 posts · tagged agents · oldest Feb 2025
- Five months, two repos, one lesson: the model was the easy partA retrospective on OpenOperator. What I would keep, what I would redo, and why a commit called "remove hardcoded API keys" landing a year late is the most honest thing in the history.agentsinfraretrospective6 min
- MCP for mouse and keyboard: a desktop as a tool serverSix tools, one resource, and a design decision I still think is underrated — the screenshot is not a tool, it is a resource. Plus the reason we composite a cursor into every capture.mcpagentspython7 min
- "It looked like it worked": grading a non-deterministic agentThe least trustworthy signal in the system is the agent reporting success. Here is the evaluator we built instead — and the four-word commit that forced a whole second repository into existence.agentstestingevaluation7 min
- Tasks as config, not codeA JSON scenario format with lifecycle hooks, so an agent test starts from the same desktop every time — and so the person writing the test does not have to be the person who wrote the agent.agentstestingarchitecture7 min
- Four agents in four weeks: from AutoGen to a hand-rolled plan-and-solve loopThe old_agents/ directory still holds all the corpses. What each generation got wrong, and why most of a computer-use agent is not acting — it is checking whether the last action worked.agentsaiarchitecture8 min
- Reading the screen: OmniParser as a serviceTurning a screenshot into a numbered list of clickable things — YOLOv8 for detection, Florence-2 for captions, OCR for text. And why it had to be a server, not a library.aiagentspython6 min
- We tried every vision model we could get our hands onMagma, Llama 3.2 Vision, Gemma 3, Phi-4, GPT-4o, Claude — including feeding a screen recording instead of stills. What actually moved the needle was not the model.aiagentsmodels5 min
- You cannot debug an agent from logsThe day we put noVNC in front of the guest, our mean time to diagnosis dropped by an order of magnitude. Failures in computer use are visual.agentsdebugginginfra4 min
- The API is the screenWhy we built an agent that uses Microsoft Teams the way a person does — and why the model turned out to be the easy part.agentsaiinfra7 min