Writing
VEXwatch demo
- Drag a div, get a diffVEX turns browser gestures into source-code changes. Three hard problems hide in that sentence — here is what each one taught me.
- The agent spawned an agent and I could not see itA subagent is a black box that spends your money. Making it inspectable took no new instrumentation at all — the events were already firing, into nothing.
- The field that was never populatedContinuing a conversation with an agent that already finished turned out to be one line of SDK config and three days of admitting my data model thought agents were disposable.
DEX
OpenOperatorwatch demo
- Four agents in four weeks: from AutoGen to a hand-rolled plan-and-solve loopThe old_agents/ directory still holds all the corpses. What each generation got wrong, and why most of a computer-use agent is not acting — it is checking whether the last action worked.
- Reading the screen: OmniParser as a serviceTurning a screenshot into a numbered list of clickable things — YOLOv8 for detection, Florence-2 for captions, OCR for text. And why it had to be a server, not a library.
- We tried every vision model we could get our hands onMagma, Llama 3.2 Vision, Gemma 3, Phi-4, GPT-4o, Claude — including feeding a screen recording instead of stills. What actually moved the needle was not the model.
- The API is the screenWhy we built an agent that uses Microsoft Teams the way a person does — and why the model turned out to be the easy part.