Computer Use at the Edge of the Statistical Precipice — Pierluca D'Oro, Programma Labs
Aug 14, 2026 · 17:28
Programma Labs founder Pierluca D'Oro argues deterministic computer-use benchmarks are gameable: a sub-megabyte replay agent that blindly replays one successful trajectory per task matches or beats the frontier model it was copied from on OSWorld and Mobile World, and pass@k is that exploit's success rate. He proposes PRISM principles and DIGIWORLD, 15 sandboxed Android apps with 3.2 million verified configurations generated by a compiler that rejects invalid combinations; multifactorial variation blocks replay exploits and exposes model fragility. Naive rollouts give around 20% true coverage instead of 95%; he prices a 4% real gap across a million tasks at $12 per mistake at hundreds of thousands of dollars a month. Programma Labs is building CUA infrastructure and hiring.
