A hospital algorithm underrates Black patients' illness
On 25 October 2019 Science published an analysis of a widely used commercial algorithm that selects patients for extra care: at the same score Black patients were considerably sicker than White patients, and removing the gap would raise their share among those selected from 17.7% to 46.5%.
Why it matters
The bias lay not in data about race but in what the model predicts: health care costs instead of illness. Because less is spent on Black patients' care, a model accurate on cost turned out biased on illness.
The authors describe the algorithm as typical of the industry and affecting millions of patients. The paper's conclusion reaches beyond medicine: a convenient proxy label in place of the real target can be a source of bias in many tasks. The abstract names neither the algorithm nor its maker, and the record claims neither; the full text on science.org was not read.