SimulationMaxxing: How we ship agents 20× faster — Aman Gupta (Nubank) + Shreya Rajpal (Snowglobe)
Jul 29, 2026 · 16:29
Shreya Rajpal (CEO of Snowglobe) and Aman (Principal ML Engineer at Nubank) argue that generating evaluation data in simulation instead of waiting on production data lets Nubank ship AI agents 20× faster. Nubank serves 135 million customers and has five agents in production, with TNPS approaching human quality. Snowglobe points at the agent, generates thousands of grounded multi-turn conversations (e.g., persona Maria Souza ordering a credit card), and pipes results into evals. Human review found simulated conversations comparable to real ones 80% of the time, enabling the team to catch regressions before production and improve one agent's self-service rate by 4%. The tight ship-observe-simulate-repeat loop also lets them test open-source models against frontier models in days instead of weeks, because the eval bottleneck is gone.
