How Evals and Prompts Shape Agent Behavior — Preetika Bhateja & Daniel Bump, YouTube Ads
Jul 24, 2026 · 19:29
Preetika Bhateja, Chris Souza, and Daniel Bump from Google share lessons from building evals for a seed-asset agent that turns messy YouTube ad creatives into clean assets. They argue that reliable agent behavior emerges from a loop of prompts, evals, iteration, and feedback — not just prompting alone. The team recommends starting small with intuition-based 'vibing' to understand failure patterns before scaling to human raters or LLM-as-judge. Using clear rubrics, obtaining rater explanations, and analyzing agent trace logs help uncover why failures occur, such as the agent removing disclaimers despite explicit instructions. They emphasize focusing on patterns across multiple examples rather than isolated failures, and investing in online evals with production data to keep evaluations representative. The takeaway: a good eval system evolves with the product, requires curated golden sets, and needs clear launch criteria to distinguish acceptable trade-offs from critical regressions.
