Self-Attention Based Multi-Model Fusion Mechanism for Diabetic Retinopathy Grading

Abstract

Among the five basic human senses, vision is the most versatile, which depends primarily on the flawless working of human eyes. An eye condition called diabetic retinopathy (DR) is caused by diabetes. Retinal lesions are important signs of DR, and they can be used for early diagnosis. This research will identify and evaluate any pathological alterations brought on by the development of DR. In this research, an attention based deep learning method is proposed to classify Diabetic Retinopathy (DR), wherein CLAHE preprocessing is first used to enhance the image contrast. The publicly accessible dataset APTOS 2019 is utilized in the experiments to evaluate the effectiveness of proposed methodology. However, this dataset has imbalanced class distribution, which was balanced using different data augmentation strategies. For effective feature learning, four pre-trained models namely Inception v3, VGG-19, ResNet-50, and DensNet-121 are modified by adding self- attention blocks following each convolutional base of the pre-trained models. These self-attention blocks enable the model to selectively focus on important parts of DR lesions. Subsequently, the extracted features are fused using late fusion, and a Multi- Layer Perceptron (MLP) classifier is employed to categorize DR stages. The proposed method was validated by performing ablation studies for each pre-trained model. Specifically, classification evaluation metrics including precision, recall, F1- score, and accuracy were computed for each pre-trained model with and without the attention block. The comparison of results clearly showed the effectiveness of adding the attention blocks to each model. Further, the final MLP classifier trained on the fused convolutional features from all pre-trained models was also evaluated for DR grading, which resulted into precision, recall, F1-score, and accuracy of 0.83%, 0.81%, 0.82% and 0.90%, respectively. These results were compared with state-of-the-art methods to show that the proposed method outperforms existing methods significantly. Therefore, the proposed method can be a useful tool for providing a secondary opinion to medical practitioners for DR grading and classification.