OpenAI Model o1 Outperforms Harvard Physicians in Emergency Triage Diagnosis Accuracy Trial
A study by Harvard researchers found that OpenAI o1 achieved a 67 percent diagnostic accuracy in emergency triage, surpassing the 50 to 55 percent range recorded by human doctors.

A recent study conducted by Harvard University researchers has demonstrated that OpenAI o1 significantly outperforms human physicians in clinical reasoning within emergency department triage. In a series of diagnostic tests involving emergency patients, the AI model achieved a 67 percent success rate. This performance notably exceeds the accuracy levels of human doctors, who typically scored between 50 and 55 percent in the same evaluation environment. Published in the journal Science, the findings suggest that large language models have reached a pivotal threshold in medical reasoning capabilities. The research indicates that these systems are now surpassing established benchmarks for clinical decision-making, particularly in high-pressure scenarios where rapid and accurate initial assessments are critical for patient outcomes. While the study highlights the transformative potential of AI in healthcare, it also underscores a shift in how medical professionals might eventually integrate machine intelligence into their workflows. The results provide a empirical basis for further exploration of LLMs as supportive tools for triage, potentially reducing diagnostic errors in emergency settings.
Primary source
Open Hacker News ↗This article summarizes the source. Check the original for full details and conditions.
Comparison
| Aspect | Before / Alternative | After / This |
|---|---|---|
| Diagnostic Accuracy | 50% to 55% (Human Doctors) | 67% (OpenAI o1) |
| Reasoning Capability | Human clinical judgment benchmarks | LLM-driven advanced clinical reasoning |
| Diagnostic Speed | Manual physician triage process | Instantaneous model-based processing |
| Primary Methodology | Standard clinical training and experience | Inference-time scaling and chain-of-thought |