Epic's sepsis model misses two thirds
On 21 June 2021 JAMA Internal Medicine published an external validation of Epic's sepsis model, deployed in hundreds of US hospitals: on 38,455 hospitalisations in Michigan its area under the ROC curve was 0.63 against 0.76-0.83 in Epic's documentation, and it missed 67% of sepsis cases.
Why it matters
A model already alerting physicians across the country had, the authors say, not been adequately evaluated, and an external test found it far worse than claimed. The authors conclude that its spread despite this raises questions about sepsis care nationally.
27,697 adult patients, sepsis in 2,552 hospitalisations (7%). The model is a penalised logistic regression on health-record data, not a deep network, developed on 405,000 encounters in three health systems. At a threshold of 6 it alerted on 18% of hospitalisations and, of 2,552 sepsis cases, found by itself 183 that had not received timely antibiotics. The authors took 0.76-0.83 from Epic's internal documentation, shared with them by permission, and from a conference paper co-written with Epic; that documentation was not read.