Evaluating the Diagnostic Accuracy and Management Recommendations of ChatGPT in Uveitis

William Rojas-Carabali, Carlos Cifuentes-González, Xin Wei, Ikhwanuliman Putera, Alok Sen, Zheng Xian Thng, Rajdeep Agrawal, Tobias Elze, Lucia Sobrin, John H. Kempen, Bernett Lee, Jyotirmay Biswas, Quan Dong Nguyen, Vishali Gupta, Alejandra de-la-Torre, Rupesh Agrawal

Research output: Contribution to journalArticlepeer-review

4 Scopus citations

Abstract

Introduction: Accurate diagnosis and timely management are vital for favorable uveitis outcomes. Artificial Intelligence (AI) holds promise in medical decision-making, particularly in ophthalmology. Yet, the diagnostic precision and management advice from AI-based uveitis chatbots lack assessment. Methods: We appraised diagnostic accuracy and management suggestions of an AI-based chatbot, ChatGPT, versus five uveitis-trained ophthalmologists, using 25 standard cases aligned with new Uveitis Nomenclature guidelines. Participants predicted likely diagnoses, two differentials, and next management steps. Comparative success rates were computed. Results: Ophthalmologists excelled (60–92%) in likely diagnosis, exceeding AI (60%). Considering fully and partially accurate diagnoses, ophthalmologists achieved 76–100% success; AI attained 72%. Despite an 8% AI improvement, its overall performance lagged. Ophthalmologists and AI agreed on diagnosis in 48% cases, with 91.6% exhibiting concurrence in management plans. Conclusions: The study underscores AI chatbots' potential in uveitis diagnosis and management, indicating their value in reducing diagnostic errors. Further research is essential to enhance AI chatbot precision in diagnosis and recommendations.

Original languageEnglish (US)
JournalOcular Immunology and Inflammation
DOIs
StatePublished - Sep 18 2023

All Science Journal Classification (ASJC) codes

  • Immunology and Allergy
  • Ophthalmology

Fingerprint

Dive into the research topics of 'Evaluating the Diagnostic Accuracy and Management Recommendations of ChatGPT in Uveitis'. Together they form a unique fingerprint.

Cite this