Chatbots Vs. Human Experts: Evaluating Diagnostic Performance of Chatbots in Uveitis and the Perspectives on AI Adoption in Ophthalmology

William Rojas-Carabali; Alok Sen; Aniruddha Agarwal; Gavin Tan; Carol Y. Cheung; Andres Rousselot; Rajdeep Agrawal; Renee Liu; Carlos Cifuentes-González; Tobias Elze; John H. Kempen; Lucia Sobrin; Quan Dong Nguyen; Alejandra de-la-Torre; Bernett Lee; Vishali Gupta; Rupesh Agrawal

doi:10.1080/09273948.2023.2266730

Chatbots Vs. Human Experts: Evaluating Diagnostic Performance of Chatbots in Uveitis and the Perspectives on AI Adoption in Ophthalmology

Título traducido de la contribución: Chatbots frente a expertos humanos: Evaluación del rendimiento diagnóstico de los chatbots en uveítis y las perspectivas de adopción de la IA en oftalmología

William Rojas-Carabali, Alok Sen, Aniruddha Agarwal, Gavin Tan, Carol Y. Cheung, Andres Rousselot, Rajdeep Agrawal, Renee Liu, Carlos Cifuentes-González, Tobias Elze, John H. Kempen, Lucia Sobrin, Quan Dong Nguyen, Alejandra de-la-Torre, Bernett Lee, Vishali Gupta, Rupesh Agrawal

Producción científica: Contribución a una revista › Artículo › revisión exhaustiva

2 Citas (Scopus)

Resumen

Propósito: Evaluar el rendimiento diagnóstico de dos chatbots, ChatGPT y Glass, en el diagnóstico de la uveítis en comparación con especialistas en uveítis de renombre, y evaluar la percepción de los clínicos sobre la utilización de la inteligencia artificial (IA) en la práctica oftalmológica.

Métodos: Se presentaron seis casos a expertos en uveítis, ChatGPT (versión 3.5 y 4.0) y Glass 1.0, y se analizó la precisión diagnóstica. Además, se realizó una encuesta sobre las emociones, la confianza en la utilización de herramientas basadas en IA y la probabilidad de incorporar dichas herramientas en la práctica clínica.

Resultados: Los expertos en uveítis diagnosticaron con precisión todos los casos (100%), mientras que ChatGPT alcanzó una tasa de acierto diagnóstico del 66% y Glass 1.0 del 33%. La mayoría de los asistentes se mostraron entusiasmados u optimistas respecto a la utilización de la IA en la práctica oftalmológica. La edad avanzada y el alto nivel educativo se correlacionaron positivamente con una mayor inclinación a adoptar herramientas basadas en IA.

Conclusiones: ChatGPT demostró capacidades de diagnóstico prometedoras en casos de uveítis y los oftalmólogos mostraron entusiasmo por la integración de la IA en la práctica clínica.

Título traducido de la contribución	Chatbots frente a expertos humanos: Evaluación del rendimiento diagnóstico de los chatbots en uveítis y las perspectivas de adopción de la IA en oftalmología
Idioma original	Inglés estadounidense
Publicación	Ocular Immunology and Inflammation
DOI	https://doi.org/10.1080/09273948.2023.2266730
Estado	Publicación electrónica previa a su impresión - oct. 13 2023

Áreas temáticas de ASJC Scopus

Inmulogía y alergología
Oftalmología

ODS de las Naciones Unidas

Este resultado contribuye a los siguientes Objetivos de Desarrollo Sostenible

Acceder al documento

10.1080/09273948.2023.2266730

Otros archivos y enlaces

Citar esto

Rojas-Carabali, W., Sen, A., Agarwal, A., Tan, G., Cheung, C. Y., Rousselot, A., Agrawal, R., Liu, R., Cifuentes-González, C., Elze, T., Kempen, J. H., Sobrin, L., Nguyen, Q. D., de-la-Torre, A., Lee, B., Gupta, V., & Agrawal, R. (2023). Chatbots Vs. Human Experts: Evaluating Diagnostic Performance of Chatbots in Uveitis and the Perspectives on AI Adoption in Ophthalmology. Ocular Immunology and Inflammation. Avanzar publicación online. https://doi.org/10.1080/09273948.2023.2266730

@article{5f21c20a6cac452ea0b8e4f4d061eda6,

title = "Chatbots Vs. Human Experts: Evaluating Diagnostic Performance of Chatbots in Uveitis and the Perspectives on AI Adoption in Ophthalmology",

abstract = "Purpose: To assess the diagnostic performance of two chatbots, ChatGPT and Glass, in uveitis diagnosis compared to renowned uveitis specialists, and evaluate clinicians{\textquoteright} perception about utilizing artificial intelligence (AI) in ophthalmology practice. Methods: Six cases were presented to uveitis experts, ChatGPT (version 3.5 and 4.0) and Glass 1.0, and diagnostic accuracy was analyzed. Additionally, a survey about the emotions, confidence in utilizing AI-based tools, and the likelihood of incorporating such tools in clinical practice was done. Results: Uveitis experts accurately diagnosed all cases (100%), while ChatGPT achieved a diagnostic success rate of 66% and Glass 1.0 achieved 33%. Most attendees felt excited or optimistic about utilizing AI in ophthalmology practice. Older age and high level of education were positively correlated with increased inclination to adopt AI-based tools. Conclusions: ChatGPT demonstrated promising diagnostic capabilities in uveitis cases and ophthalmologist showed enthusiasm for the integration of AI into clinical practice.",

author = "William Rojas-Carabali and Alok Sen and Aniruddha Agarwal and Gavin Tan and Cheung, {Carol Y.} and Andres Rousselot and Rajdeep Agrawal and Renee Liu and Carlos Cifuentes-Gonz{\'a}lez and Tobias Elze and Kempen, {John H.} and Lucia Sobrin and Nguyen, {Quan Dong} and Alejandra de-la-Torre and Bernett Lee and Vishali Gupta and Rupesh Agrawal",

note = "Publisher Copyright: {\textcopyright} 2023 Taylor & Francis Group, LLC.",

year = "2023",

month = oct,

day = "13",

doi = "10.1080/09273948.2023.2266730",

language = "English (US)",

journal = "Ocular Immunology and Inflammation",

issn = "0927-3948",

publisher = "Informa Healthcare",

}

Rojas-Carabali, W, Sen, A, Agarwal, A, Tan, G, Cheung, CY, Rousselot, A, Agrawal, R, Liu, R, Cifuentes-González, C, Elze, T, Kempen, JH, Sobrin, L, Nguyen, QD, de-la-Torre, A, Lee, B, Gupta, V & Agrawal, R 2023, 'Chatbots Vs. Human Experts: Evaluating Diagnostic Performance of Chatbots in Uveitis and the Perspectives on AI Adoption in Ophthalmology', Ocular Immunology and Inflammation. https://doi.org/10.1080/09273948.2023.2266730

TY - JOUR

T1 - Chatbots Vs. Human Experts

T2 - Evaluating Diagnostic Performance of Chatbots in Uveitis and the Perspectives on AI Adoption in Ophthalmology

AU - Rojas-Carabali, William

AU - Sen, Alok

AU - Agarwal, Aniruddha

AU - Tan, Gavin

AU - Cheung, Carol Y.

AU - Rousselot, Andres

AU - Agrawal, Rajdeep

AU - Liu, Renee

AU - Cifuentes-González, Carlos

AU - Elze, Tobias

AU - Kempen, John H.

AU - Sobrin, Lucia

AU - Nguyen, Quan Dong

AU - de-la-Torre, Alejandra

AU - Lee, Bernett

AU - Gupta, Vishali

AU - Agrawal, Rupesh

PY - 2023/10/13

Y1 - 2023/10/13

N2 - Purpose: To assess the diagnostic performance of two chatbots, ChatGPT and Glass, in uveitis diagnosis compared to renowned uveitis specialists, and evaluate clinicians’ perception about utilizing artificial intelligence (AI) in ophthalmology practice. Methods: Six cases were presented to uveitis experts, ChatGPT (version 3.5 and 4.0) and Glass 1.0, and diagnostic accuracy was analyzed. Additionally, a survey about the emotions, confidence in utilizing AI-based tools, and the likelihood of incorporating such tools in clinical practice was done. Results: Uveitis experts accurately diagnosed all cases (100%), while ChatGPT achieved a diagnostic success rate of 66% and Glass 1.0 achieved 33%. Most attendees felt excited or optimistic about utilizing AI in ophthalmology practice. Older age and high level of education were positively correlated with increased inclination to adopt AI-based tools. Conclusions: ChatGPT demonstrated promising diagnostic capabilities in uveitis cases and ophthalmologist showed enthusiasm for the integration of AI into clinical practice.

AB - Purpose: To assess the diagnostic performance of two chatbots, ChatGPT and Glass, in uveitis diagnosis compared to renowned uveitis specialists, and evaluate clinicians’ perception about utilizing artificial intelligence (AI) in ophthalmology practice. Methods: Six cases were presented to uveitis experts, ChatGPT (version 3.5 and 4.0) and Glass 1.0, and diagnostic accuracy was analyzed. Additionally, a survey about the emotions, confidence in utilizing AI-based tools, and the likelihood of incorporating such tools in clinical practice was done. Results: Uveitis experts accurately diagnosed all cases (100%), while ChatGPT achieved a diagnostic success rate of 66% and Glass 1.0 achieved 33%. Most attendees felt excited or optimistic about utilizing AI in ophthalmology practice. Older age and high level of education were positively correlated with increased inclination to adopt AI-based tools. Conclusions: ChatGPT demonstrated promising diagnostic capabilities in uveitis cases and ophthalmologist showed enthusiasm for the integration of AI into clinical practice.

UR - http://www.scopus.com/inward/record.url?scp=85174047282&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85174047282&partnerID=8YFLogxK

U2 - 10.1080/09273948.2023.2266730

DO - 10.1080/09273948.2023.2266730

M3 - Article

C2 - 37831553

AN - SCOPUS:85174047282

SN - 0927-3948

JO - Ocular Immunology and Inflammation

JF - Ocular Immunology and Inflammation

ER -

Chatbots Vs. Human Experts: Evaluating Diagnostic Performance of Chatbots in Uveitis and the Perspectives on AI Adoption in Ophthalmology

Resumen

Áreas temáticas de ASJC Scopus

ODS de las Naciones Unidas

Acceder al documento

Otros archivos y enlaces

Huella

Citar esto