Unsupervised method to cluster color fundus eye images and text reports from patients with diabetic retinal lesions

Abstract
Purpose : A utility of Artificial Intelligence (AI) aided diagnosis on medical information is the identification and extraction of relevant information in an unsupervised fashion. Several AI studies have focused in the ability of different methods to classify based on graphic data, but most medical information is based on text data. Color fundus eye image is one of the most used data by the retina specialist, commonly these images have its corresponding text report. The purpose of this study was to evaluate the ability of an unsupervised method to cluster color fundus eye images and the text reports

Methods : In a cross-sectional study, one-hundred images from patients with diabetic retinopathy and/or diabetic macular edema were analyzed by a retinal specialist. A text report was created from each image. The size of the images was adjusted to 224 x 224 pixels and the text reports were processed using the library in NLTK for Python text. A designed end-to-end method based on deep learning algorithm was tested with the images and its corresponding text reports. The ability to cluster the images and the text reports according to the diabetic retinal lesions was measured with k-means

Results : The unsupervised method identified a 5604 possible word combinations in the text reports (as an unigrams, bigrams and trigrams). Distance in the k-means analysis (figure 1) showed that the group numbers 4 and 5 were the best fit to cluster the color fundus eye images. Likewise, the distance in the k-means analysis (figure 2) showed that the group numbers 3, 4 and 5 were the best fit to cluster the text reports. There was a correspondence between the group numbers identified for clustering the images and its text reports. Group numbers 4 and 5 had a high similarity in diabetic retinal lesions, in both text and images reports