Caution Urged as AI Chatbots Gain Popularity in Healthcare
Have you ever used ChatGPT-4 or another AI chatbot to ask a health-related question? Maybe your doctor relies on ChatGPT-4 to summarize your visits or even double-check their diagnosis. While these AI chatbots may seem reliable, experts are warning both consumers and doctors to proceed with caution. The accuracy of AI chatbots is not always guaranteed.
Calls for Government Regulation
As the use of AI chatbots continues to grow in healthcare and other industries, there is a growing demand for government regulation to protect the public from potential unintended consequences. President Joe Biden recently issued an executive order that directs government agencies to develop ways to govern the use of AI. In healthcare, the Department of Health and Human Services is tasked with advancing responsible AI innovation that prioritizes patient and worker welfare.
Establishing Regulations and Monitoring
The Department of Health and Human Services is expected to establish a health care AI task force within a year. This task force will develop a plan to regulate the use of AI in healthcare delivery, public health, and research and development. The plan will also address long-term safety and real-world performance monitoring of AI technologies. Additionally, the department will collaborate with other agencies and patient safety organizations to identify errors resulting from AI use in clinical settings.
Experts React to the Executive Order
Ida Sim, MD, PhD, a professor at the University of California, San Francisco, views Biden’s executive order as a positive first step. John W. Ayers, PhD, from the University of California San Diego, agrees that regulation is necessary but emphasizes the need to balance it with medical progress.
The “Hallucination” Issue
AI chatbots like ChatGPT-4 have become prevalent in healthcare, with 14% of doctors already using them for various purposes. However, these chatbots are not always accurate. Studies have shown that chatbots can invent information, with rates ranging from 3% to 27% depending on the bot. While they often provide correct answers, there are instances where they deliver completely incorrect information.
The Accuracy of Chatbots
In a trial involving 33 doctors from different specialties, chatbots answered medical questions with nearly correct or completely correct responses more than half of the time. However, there were 15 questions where the answers were scored as completely incorrect. Google’s chatbot, Med-PaLM, designed specifically for medical knowledge, achieved an accuracy rate of 92.6% in answering medical questions.
Comparing Chatbots and Doctors
A study comparing chatbot responses to doctors’ responses found that health professionals preferred the chatbot responses in nearly 80% of the exchanges. Doctors’ answers were rated lower in terms of quality and empathy. The researchers suggested that doctors’ stress levels might have affected their responses.
The Challenge of Incomplete Data
One challenge for chatbots is the presence of incorrect or missing information in electronic health records. Diagnostic errors often occur due to incomplete patient histories and inadequate physical exams documented in these records. Additionally, the records may not include information from other practitioners who have seen the patient. In such cases, a doctor’s experience and knowledge of the patient become crucial in drawing the right conclusion.
The Potential of Chatbots
Despite their limitations, chatbots excel in communicating with patients. With human supervision, they can help alleviate the burden of online messaging for doctors and improve the quality of care. Chatbots can send personalized messages to patients based on their records and anticipated needs, potentially transforming how patients interact with healthcare providers.
Managing Chronic Diseases
Chatbots can also play a significant role in managing chronic diseases, which affect a large portion of the population. They can provide support, analyze data from monitoring sensors, and ask patients about their condition on a daily basis. However, rigorous trials are necessary to determine whether AI-assisted messaging services can genuinely improve patient outcomes.
Conclusion
In the era of burgeoning AI chatbot integration in healthcare, caution emerges as a crucial watchword. Despite the increasing prevalence of AI chatbots like ChatGPT-4 in medical settings, concerns persist about their reliability, accuracy, and potential for misinformation. The call for government regulation gains momentum, with President Joe Biden’s executive order prompting the establishment of a healthcare AI task force within the Department of Health and Human Services. As the healthcare landscape evolves, the delicate balance between regulatory measures and technological advancement becomes paramount. While chatbots exhibit promise in communication and chronic disease management, the imperative lies in rigorous trials to ascertain their true potential and ensure optimal patient outcomes.
Photo: Freepik.com
