AI can tell a patient's race from an X-ray, nobody knows how

A multi-institution team including researchers at Emory University trained standard deep learning models on chest X-rays, mammograms, and CT scans, and asked a plain question: could the model guess a patient's self-reported race from the image alone.

Across imaging types and hospital systems, the models identified race with AUC scores of 0.81 to 0.99, holding up even after the images were degraded, cropped, or blurred past what a radiologist could read. Removing obvious clues changed nothing.

The researchers could not find the signal themselves. It was not bone density, body size, or any feature they could isolate and remove, and it survived manipulations that should have erased it. The finding raised an open worry: if race is encoded invisibly in medical images, an algorithm trained for something else could end up discriminating by race without anyone programming it to.

A signal nobody can see, and nobody can explain.