The Illusion of Understanding: A Randomized Controlled Trial of LLM-Generated Lay Summaries of Brain MRI Reports

The study examined whether summaries generated by an artificial intelligence language model (Mistral Small 3.2) improve patient understanding of brain MRI reports. The research involved 2,727 adult participants who interpreted six standardized brain MRI reports related to headaches. Half received reports in their original format, while the other half received reports with an added lay summary. Results showed that summaries did not improve objective patient comprehension (58.3% versus 59.4%, with no statistically significant difference). However, summaries significantly improved patient satisfaction (64.9% versus 36.7%) and subjective comprehension (50.3% versus 24.0%). The study revealed a gap between how patients perceived their understanding and their actual understanding of the reports. The authors concluded that this discrepancy needs to be addressed before clinical implementation.