Evals-Driven Development for a Mental Health AI Coach — Akele Reed & Dave Revere, SonderMind
Jul 25, 2026 · 21:17
SonderMind engineers Akele Reed and Dave Revere explain how they built Sonder, a clinically grounded Mental Health AI Coach, using eval-driven development and modular guardrails to balance effectiveness and safety. They designed input and output guardrails as separate LLM judges to avoid over-calibration and ensure correct triggers, not more triggers. Dave describes a clinical feedback loop where therapist annotations become typed evals that gate releases, turning clinician judgment into CI. They open-sourced 200 input and 100 output guardrail scenarios, clinically reviewed and calibrated. The system uses a Supervisor/Executor/Evaluator architecture, and they turned off built-in guardrails of frontier models due to over-calibration. Every architectural decision prioritized user safety, with modularity enabling iteration without compromising safety.
