NEJM AI Study: 300 Neurosurgical Patients in First Known RCT Using Claude as Intervention

Details

A randomized controlled trial involving 300 neurosurgical patients, described by the senior author as the first published RCT with Claude as an intervention in patient care to their knowledge.
Zero factual errors, harmful statements, or corrections were recorded across the trial, with an interrater reliability (ICC) of 0.74.
Consultations were a median 8 minutes shorter (25 vs. 33 minutes), and patients rated the interaction better, particularly regarding shared decision-making.
Physicians reported slightly lower autonomy support and found encounters more demanding as patients arrived prepared with harder questions.
Summaries were generated by Claude 3.7 Sonnet via API with temperature 0 and a frozen prompt (version 28), targeting B1 German reading level.
Quality control involved four parallel LLM judges from OpenAI o1 (scoring 0-10) and mandatory physician review against source documents.
The author notes the study is a proof of concept, not practice changing, with limitations including single-center design, lack of blinding, no clinical outcome measures, and restriction to German speakers.

Key point