A ground-breaking international study led by researchers from 黑料网大事记 Sydney and the University of Melbourne has revealed both the potential and pitfalls of using generative AI in healthcare.
Published in npj Digital Medicine, the research evaluated Baidu鈥檚 ERNIE Bot鈥擟hina鈥檚 leading AI chatbot鈥攆or its ability to diagnose and manage chronic diseases in low-resource settings.
Using simulated patient experiments, the team conducted 384 trials involving two common conditions: unstable angina and asthma. ERNIE Bot demonstrated high diagnostic accuracy (77.3%) and correct medication prescriptions (94.3%), outperforming human primary care providers in China. However, the chatbot also showed a concerning tendency to overprescribe, with 91.9% of consultations involving unnecessary lab tests and 57.8% involving inappropriate medications.
The study also uncovered disparities in care based on patient age and socioeconomic status, with older and wealthier patients receiving more intensive, and often excessive treatment. Comparisons with other AI models, including ChatGPT-4o and DeepSeek R1, revealed similar patterns of high accuracy but low adherence to clinical checklists and elevated safety risks.
Lead author Dr Yafei Si emphasised the importance of rigorous, context-specific evaluation before integrating AI into healthcare systems. 鈥淎I chatbots like ERNIE Bot could revolutionise access to care in underserved regions,鈥 he said, 鈥渂ut safeguards are essential to ensure equity, safety, and accountability.鈥
The research highlights the urgent need for ethical design, stakeholder engagement, and human oversight in deploying AI tools for clinical decision-making, especially in low and middle income countries.