Agentic Evaluation Pipeline for Medical AI
An agentic triage layer that cuts expert annotation cost while preserving clinical grounding (ARPA-H CARE).
As part of ARPA-H’s CARE program, I lead the technical execution of a safety-evaluation suite for patient-facing medical chatbots, coordinating 10+ clinicians, engineers, and graduate students.
At its core is an agentic evaluation pipeline (LangChain, Langfuse) that triages AI-flagged false positives and false negatives before they reach a clinician. The goal: meaningfully reduce expert annotation burden while preserving clinical grounding, so that scarce clinician time is spent only where automated judgment is genuinely uncertain.