Researchers at NYU Langone Health built a language model called NYUTron, trained on more than 10 years of the hospital's own clinical notes, to see whether the plain language doctors already write could predict outcomes better than the usual checklist of lab values and diagnosis codes.
The model read roughly 10 years of notes representing hundreds of thousands of patients, then was tested on predicting 30-day readmission, in-hospital death and other outcomes. On readmission it scored 79.9 percent on a standard accuracy measure, the area under the curve, about 5 points better than traditional models.
NYUTron was trained entirely on the words doctors already type into a chart, not new tests or scans, which makes the gain notable: the information needed was sitting in plain text the whole time, unused by older prediction tools built around structured numbers. The model is now running inside NYU Langone hospitals, one of the first hospital-trained language models used on live patients day to day.
Read the original study
Jiang et al., Nature, 2023 · doi.org/10.1038/s41586-023-06160-y
The chart was the signal all along.
Source
- Jiang, L. Y., Liu, X. C., Nejatian, N. P., Nasir-Moin, M., Wang, D., Abidin, A., Eaton, K., Riina, H. A., Laufer, I., Punjabi, P., Miceli, M., Kim, N. C., Orillac, C., Schnurman, Z., Livia, C., Weiss, H., Kurland, D., Neifert, S., Dastagirzada, Y., . . . Oermann, E. K. (2023). Health system-scale language models are all-purpose prediction engines. Nature, 619(7969), 357–362. https://doi.org/10.1038/s41586-023-06160-y