A small study meant to see if doctors were able to improve diagnoses with the help of ChatGPT yielded unsettling results : the AI chatbot was better

ChatGPT beat doctors at diagnosing medical conditions, study says

submited by

Style Pass

2024-11-25 04:30:04

A small study meant to see if doctors were able to improve diagnoses with the help of ChatGPT yielded unsettling results : the AI chatbot was better at diagnosis than the doctors themselves.

The study asked 50 doctors, 26 of whom were attending physicians and 24 of whom were residents, to try to make six different diagnoses for medical conditions with the same case histories. Some doctors were given OpenAI’s ChatGPT to help them make their decisions; others went without AI. Doctors who did the project without AI got an average score of 74%, doctors who used AI got an average score of 76%, and ChatGPT itself got an average score of 90%.

Dr. Rodman, who helped design the study, told the New York Times he expected the chatbots would significantly help doctors using them. He was “shocked” to see it made little difference and shocked again that the AI itself beat the doctors.

AI didn’t help doctors using it as much as anticipated because physicians “didn’t listen to AI when AI told them things they didn’t agree with,” Rodman said. Most doctors, he said, were wedded to their own diagnoses and couldn’t be convinced a chatbot knew more than them.