OpenOperator
computer-use agents on real Windows VMs, in Kubernetes
2026
2025
- Benchmarking AI agents: node pools, job queues, and cost per stepFive workload classes with five different scaling shapes, a bounded queue that fans tests into Kubernetes Jobs, and the reason your autoscaler makes terrible decisions about 30 GB virtual machines.
- One proxy pod per VM does not scaleA design that is obviously right at n=1 and obviously wrong at n=20, with no warning in between. The commit that deleted it is the best one in the repository.
- Streaming helm upgrade to a browserProvisioning a VM takes three minutes. A spinner is not an acceptable answer. An async generator, a path-keyed WebSocket relay, and one design decision I would not repeat.
- Forty minutes to ninety seconds: VolumeSnapshots as golden images for VMsOne Helm chart, two completely different storage topologies behind a single boolean. Plus the two bugs it cost — a StatefulSet naming convention and a substring match that matched too much.
- Decoding Microsoft Bond: reading an app’s own telemetry through mitmproxyGround truth from an application with no test hooks. mitmproxy local mode against a single process, a hand-written Bond Compact Binary decoder, and the one-line AKS bug that killed a day.
- Shipping a 6 GB ISO to every pod (and then deleting the storage key)An initContainer that pulls once, one PVC projected into eleven mount points via subPath, and a three-month migration from a storage key to workload identity.
- Nested virtualisation on AKS: yes, you can run KVM in a podhostPath /dev/kvm, privileged: true, and the Deployment→StatefulSet migration. Plus the annotation that stops the autoscaler evicting a forty-minute Windows install.
- MCP for mouse and keyboard: a desktop as a tool serverSix tools, one resource, and a design decision I still think is underrated — the screenshot is not a tool, it is a resource. Plus the reason we composite a cursor into every capture.
- "It looked like it worked": grading a non-deterministic agentThe least trustworthy signal in the system is the agent reporting success. Here is the evaluator we built instead — and the four-word commit that forced a whole second repository into existence.
- Tasks as config, not codeA JSON scenario format with lifecycle hooks, so an agent test starts from the same desktop every time — and so the person writing the test does not have to be the person who wrote the agent.
- Four agents in four weeks: from AutoGen to a hand-rolled plan-and-solve loopThe old_agents/ directory still holds all the corpses. What each generation got wrong, and why most of a computer-use agent is not acting — it is checking whether the last action worked.
- The bootstrap from hell: provisioning a Windows guest you cannot SSH intoNo SSH, no cloud-init, no config management. Just a .bat file, an SMB share the hypervisor provides, and one Task Scheduler flag that decides whether your screenshots are black.
- Reading the screen: OmniParser as a serviceTurning a screenshot into a numbered list of clickable things — YOLOv8 for detection, Florence-2 for captions, OCR for text. And why it had to be a server, not a library.
- We tried every vision model we could get our hands onMagma, Llama 3.2 Vision, Gemma 3, Phi-4, GPT-4o, Claude — including feeding a screen recording instead of stills. What actually moved the needle was not the model.
- You cannot debug an agent from logsThe day we put noVNC in front of the guest, our mean time to diagnosis dropped by an order of magnitude. Failures in computer use are visual.
- QEMU in Docker: a real Windows 11 VM as a containerIt is not Windows in a container. It is QEMU in a container with a Windows ISO and an unattended answer file — and the difference is three lines of compose.yml.
- The API is the screenWhy we built an agent that uses Microsoft Teams the way a person does — and why the model turned out to be the easy part.