AI Evals for Cross-Functional Teams — Nachiket Paranjape & Swaroop Chitlur Haridas, DoorDash
Aug 28, 2026 · 16:11
DoorDash GenAI platform team's Swaroop Chitlur Haridas and Nachiket Paranjape argue evals stopped being an engineering harness and became a cross-functional effort spanning strategy and operations, product, operations, and engineering. They describe a continuous loop — trace, sample, annotate, calibrate — and an API-first platform that lets non-engineers use Codex or Claude Code to vibe code their own annotation UIs. Judge prompts are calibrated self-serve through a UI showing original and optimized prompts side by side, using JetPa, so product managers and operators can run optimization loops without engineering. Per-annotation cost fell sharply at DoorDash scale, and variation in who owns judge prompts across teams is treated as a sign the org is still learning.
