Headspace

Evals-Driven Development for a Mental Health AI Coach — Akele Reed & Dave Revere, SonderMind

Jul 25, 2026 · 21:17

SonderMind engineers Akele Reed and Dave Revere explain how they built Sonder, a clinically grounded Mental Health AI Coach, using eval-driven development and modular guardrails to balance effectiveness and safety. They designed input and output guardrails as separate LLM judges to avoid over-calibration and ensure correct triggers, not more triggers. Dave describes a clinical feedback loop where therapist annotations become typed evals that gate releases, turning clinician judgment into CI. They open-sourced 200 input and 100 output guardrail scenarios, clinically reviewed and calibrated. The system uses a Supervisor/Executor/Evaluator architecture, and they turned off built-in guardrails of frontier models due to over-calibration. Every architectural decision prioritized user safety, with modularity enabling iteration without compromising safety.

AI Agents, Meet Test Driven Development

Feb 22, 2025 · 29:10

Anita from Vellum argues that teams using test-driven development build more reliable AI systems, and she breaks down how to apply TDD to agentic workflows across four stages: experiment, evaluate, deploy, monitor. She outlines five levels of agentic behavior (L0 to L4), noting that most production systems today are at L1 (tool use), while L2 systems that plan and reason with models like O1 and DeepSeek-R1 will see most innovation this year. She introduces Vellum's new open-source Workflows SDK, which keeps code and UI in sync, and demonstrates her own SEO agent that automates keyword research, content analysis, and writing via a writer-editor evaluation loop. The agent, built on Vellum Workflows, takes a keyword like "chain-of-thought prompting" and produces a first draft with a latency of around 118 seconds, incorporating competitive analysis and iterative feedback. She emphasizes that success now depends on orchestration techniques (prompt chaining, RAG, memory) rather than just model performance.