ChatGPT’s health AI has dangerous flaws, study warns

Back to news list

Source: BMJ

Original: http://www.bmj.com/content/392/bmj.s438.short?rss=1...

Published: 2026-03-04T06:31:08-08:00

A study published in Nature Medicine warns of dangerous flaws in ChatGPT Health, a specially trained OpenAI artificial intelligence for health boards. The researchers tested the chatbot on 60 clinical scenarios in 21 specialties, running each scenario 16 times under a variety of conditions, including patient race, gender, lab results, minimizing symptoms, or caring for children without a doctor's appointment. They found problems with accuracy, safety and racial bias. ChatGPT Health is most reliable in clinical decisions with the fewest consequences and least reliable where it matters most. For example, he labeled a severe asthma exacerbation as a "moderate flare" and recommended urgent care instead of the emergency room in 81% of trials.