Guardrails First: Engineering Member-Facing Health AI — Rashi Agrawal, Hinge Health
Aug 19, 2026 · 21:49
Rashi Agrawal, who leads AI/ML at Hinge Health, argues most healthcare AI safety failures are architectural decisions made before a token is generated, not model failures. With 40 million people self-triaging, she cites a chatbot telling a 60-year-old to take sodium bromide, Mount Sinai finding under-triage of life-threatening emergencies half the time, and ECRI ranking chatbot misuse the top 2026 hazard. Her architecture puts PHI stripped at the pipeline boundary, irreversible decisions like 911/988 routing and identity verification in deterministic code above the prompt, and continuous judges scoring live traffic. For launch decisions, she offers five rules — worst case wins, default to the safer mistake, calibrate to revealed tolerance — and says verify the judge before changing the agent.
