Application of Neuroimaging-Based Deep Learning Model Interpretability Methods in Alzheimer′s Disease Recognition
Wang Jiarong1,2, Ke Ming1, Dong Zhanguo1,2, Wang Lubin2, Li Liang2*
1(School of Computer and Communication, Lanzhou University of Technology, Lanzhou 730050, China) 2(Institute of Military Cognition and Brain Sciences, Academy of Military Medical Sciences, Beijing 100850, China)
Abstract：Alzheimer′s disease (AD) is a neurodegenerative disease. So far the pathogenesis has not been completely understood, and it can cause severe life and economic burden to the family and society after the onset. Timely prevention and intervention can delay the occurrence and development of AD. There is no complete cure, so early screening for AD is of great significance. This review paper firstly summarized the AD prediction model based on deep learning technology and compared and analyzed the model structure, data scale, and local and global brain regions. Among them, the 3D convolutional neural network model showed the best performance, and the continuous expansion of the data scale was helpful to improve the model performance. This paper also summarized the interpretability methods of the current neuroimage model, analyzed the advantages and disadvantages of the interpretability methods based on sensitivity analysis and backpropagation in the application of AD diagnosis, and demonstrated the interpretability method represented by backpropagation was more suitable for AD research. Finally, according to the research status, next development direction was proposed, which was suggested to realize semantic medical image analysis and generate understandable diagnostic reports.
王佳荣, 柯铭, 董占国, 王路斌, 李椋. 基于神经影像的深度学习模型可解释性方法在阿尔茨海默病识别中的应用[J]. 中国生物医学工程学报, 2023, 42(4): 475-485.
Wang Jiarong, Ke Ming, Dong Zhanguo, Wang Lubin, Li Liang. Application of Neuroimaging-Based Deep Learning Model Interpretability Methods in Alzheimer′s Disease Recognition. Chinese Journal of Biomedical Engineering, 2023, 42(4): 475-485.
[1] 曾安, 贾龙飞, 潘丹, 等. 基于卷积神经网络和集成学习的阿尔茨海默症早期诊断 [J]. 生物医学工程学杂志, 2019, 36(5): 711-719. [2] 邹超. 基于三维卷积神经网络和脑网络分析的AD早期辅助诊断研究 [D]. 广州: 广东工业大学, 2020. [3] 沙小琴. 深度学习在基于fMRI图像的阿尔兹海默症分类预测中的应用研究 [D]. 保定: 河北大学, 2021. [4] 熊春文, 陈辉. 人口变迁与教育变革——基于第七次全国人口普查公报的社会学思考 [J]. 教育研究, 2021, 42(11): 27-35. [5] Li Haitao, Yuan Shaoxun, Wu Jiansheng, et al. Predicting conversion from MCI to AD combining multi-modality data and based on molecular subtype [J]. Brain Sci, 2021, 11(6): 674. [6] Pipe JG. High-value MRI.[J]. Journal of Magnetic Resonance Imaging, 2019,49(7):e12-e13. [7] 林岚, 王婧璇, 付振荣, 等. 脑老化中脑年龄预测模型研究综述 [J]. 生物医学工程学杂志, 2019, 36(3): 493-498. [8] Liu Manhua, Cheng Danni, Wang Kundong, et al. Multi-modality cascaded convolutional neural networks for Alzheimer′s disease diagnosis [J]. Neuroinformatics, 2018, 16(3-4): 295-308. [9] Li Ang, Zalesky A, Yue Weihua, et al. A neuroimaging biomarker for striatal dysfunction in schizophrenia [J]. Nat Med, 2020, 26(4): 558-565. [10] Sarvamangala DR, Kulkarni RV. Convolutional neural networks in medical image understanding: a survey[J]. Evolutionary Intelligence, 2022, 15(1): 1-22. [11] Tjoa E, Guan Cuntai. A survey on explainable artificial intelligence (XAI): toward medical XAI [J]. IEEE Trans Neural Netw Learn Syst, 2021, 32(11): 4793-4813. [12] Frisoni GB, Fox NC, Jack CR, et al. The clinical use of structural MRI in Alzheimer disease [J]. Nat Rev Neurol, 2010, 6(2): 67-77. [13] Wen J, Thibeau-Sutre E, Diaz-Melo M, et al. Convolutional neural networks for classification of Alzheimer′s disease: overview and reproducible evaluation [J]. Med Image Anal, 2020, 63: 101694. [14] Feng W, Halm-Lutterodt NV, Tang H, et al. Automated MRI-based deep learning model for detection of Alzheimer′s disease process [J]. International Journal of Neural Systems, 2020, 30(6): 2050032. [15] Bashyam VM, Erus G, Doshi J, et al. MRI signatures of brain age and disease over the lifespan based on a deep brain network and 14 468 individuals worldwide [J]. Brain, 2020, 143(7): 2312-2324. [16] He Sheng, Grant PE, Ou Yangming. Global-local transformer for brain age estimation [J]. IEEE Trans Med Imaging, 2022, 41(1): 213-224. [17] Wood DA, Kafiabadi S, Al Busaidi A, et al. Deep learning models for triaging hospital head MRI examinations[J]. Medical Image Analysis, 2022, 78: 10239. [18] Song Xiaofan, Mao Mingyi, Qian Xiaohua. Auto-metric graph neural network based on a meta-learning strategy for the diagnosis of Alzheimer′s disease [J]. IEEE J Biomed Health Inform, 2021, 25(8): 3141-3152. [19] Kim M, Kim J, Qu J, et al. Interpretable temporal graph neural network for prognostic prediction of Alzheimer’s disease using longitudinal neuroimaging data[C]//2021 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). Houston: IEEE, 2021: 1381-1384. [20] Basaia S, Agosta F, Wagner L, et al. Automated classification of Alzheimer′s disease and mild cognitive impairment using a single MRI and deep neural networks [J]. Neuroimage Clin, 2019, 21:101645. [21] Thibeau-Sutre E, Colliot O, Dormont D, et al. Visualization approach to assess the robustness of neural networks for medical image classification[C]//Medical Imaging 2020: Image Processing. Houston: SPIE, 2020, 11313: 370-380. [22] Li Hongming, Habes M, Wolk DA, et al. A deep learning model for early prediction of Alzheimer′s disease dementia based on hippocampal magnetic resonance imaging data [J]. Alzheimers Dement, 2019, 15(8): 1059-1070. [23] Lian Chunfeng, Liu Mingxia, Zhang Jun, et al. Hierarchical fully convolutional network for joint atrophy localization and Alzheimer′s disease diagnosis using structural MRI [J]. IEEE Trans Pattern Anal Mach Intell, 2020, 42(4): 880-893. [24] Lin Weiming, Tong Tong, Gao Qinhua, et al. Convolutional neural networks-based MRI image analysis for the Alzheimer′s disease prediction from mild cognitive impairment [J]. Front Neurosci, 2018, 12:777. [25] Böhle M, Eitel F, Weygandt M, et al. Layer-wise relevance propagation for explaining deep neural network decisions in MRI-based Alzheimer′s disease classification[J]. Frontiers in Aging Neuroscience, 2019, 11: 194. [26] Dyrba M, Pallath AH, Marzban EN. Comparison of CNN visualization methods to aid model interpretability for detecting Alzheimer’s disease[M]//Bildverarbeitung für die Medizin 2020. Wiesbaden: Springer Vieweg, 2020: 307-312. [27] Qiu Shangran, Joshi PS, Miller MI, et al. Development and validation of an interpretable deep learning framework for Alzheimer′s disease classification [J]. Brain, 2020, 143(6): 1920-1933. [28] Spasov S, Passamonti L, Duggento A, et al. A parameter-efficient deep learning approach to predict conversion from mild cognitive impairment to Alzheimer′s disease[J]. Neuroimage, 2019, 189: 276-287. [29] Wee CY, Liu Chaoqiang, Lee A, et al. Cortical graph neural network for AD and MCI diagnosis and transfer learning across populations [J]. Neuroimage Clin, 2019, 23:101929. [30] Wang Zijie, Turko R, Shaikh O, et al. CNN explainer: learning convolutional neural networks with interactive visualization [J]. IEEE Trans Vis Comput Graph, 2021, 27(2): 1396-1406. [31] Karimi D, Jaimes C, Machado-Rivas F, et al. Deep learning-based parameter estimation in fetal diffusion-weighted MRI [J]. Neuroimage, 2021, 243: 118482. [32] 徐金才, 任民, 李琦, 等. 图像对抗样本的安全性研究概述 [J]. 信息安全研究, 2021, 7(4): 294-309. [33] Peterson GE. Foundation for neural network verification and validation[C]// Science of Artificial Neural Networks II. Bellingham: SPIE, 1966: 196-207. [34] Huysmans J, Dejaeger K, Mues C, et al. An empirical evaluation of the comprehensibility of decision table, tree and rule based predictive models[J]. Decision Support Systems, 2011, 51(1): 141-154. [35] Ribeiro MT, Singh S, Guestrin C. " Why should I trust you?" explaining the predictions of any classifier[C]//Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. San Francisco: ACM SIGKDD, 2016: 1135-1144. [36] Zeiler MD, Fergus R. Visualizing and understanding convolutional networks[C]//European Conference on Computer Vision. Zurich:Springer, Cham, 2014: 818-833. [37] Panwar H, Gupta PK, Siddiqui MK, et al. A deep learning and grad-CAM based color visualization approach for fast detection of COVID-19 cases using chest X-ray and CT-Scan images [J]. Chaos Solitons Fractals, 2020, 140: 110190. [38] Štern D, Payer C, Urschler M. Automated age estimation from MRI volumes of the hand [J]. Med Image Anal, 2019, 58: 101538. [39] Hu Mengjiao, Qian Xing, Liu Siwei, et al. Structural and diffusion MRI based schizophrenia classification using 2D pretrained and 3D naive Convolutional Neural Networks [J]. Schizophrenia Research, 2022, 243: 330-341. [40] Natekar P, Kori A, Krishnamurthi G. Demystifying brain tumor segmentation networks: interpretability and uncertainty analysis [J]. Front Comput Neurosci, 2020, 14: 6. [41] Zhang Yu, Tiño P, Leonardis A,et al. A survey on neural network interpretability[J]. IEEE Transactions on Emerging Topics in Computational Intelligence,2021,5(5):726-742. [42] Singh A, Sengupta S, Lakshminarayanan V. Explainable deep learning models in medical image analysis[J]. Journal of Imaging, 2020, 6(6): 52-70. [43] Salahuddin Z, Woodruff HC, Chatterjee A, et al. Transparency of deep neural networks for medical image analysis: a review of interpretability methods [J]. Comput Biol Med, 2021, 140:105111. [44] Ying Zhitao, Bourgeois D, You J, et al. Gnnexplainer: Generating explanations for graph neural networks[J]. Advances in Neural Information Processing Systems, 2019, 32: 9240-9251. [45] Fong RC, Vedaldi A. Interpretable explanations of black boxes by meaningful perturbation[C]//Proceedings of the IEEE International Conference on Computer Vision. Piscataway: IEEE, 2017: 3429-3437. [46] Esmaeilzadeh S, Belivanis DI, Pohl KM, et al. End-to-end Alzheimer’s disease diagnosis and biomarker identification[C]//International Workshop on Machine Learning in Medical Imaging. Granada: Springer, Cham, 2018: 337-345. [47] Springenberg JT, Dosovitskiy A, Brox T, et al.Striving for simplicity: The all convolutional net[C]// 2015 International Conference on Learning Representations (ICLR). San Diego: ICLR, 2015: 1-14. [48] Zhou B, Khosla A, Lapedriza A, et al. Learning deep features for discriminative localization[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 2921-2929. [49] Selvaraju RR, Cogswell M, Das A, et al. Grad-cam: visual explanations from deep networks via gradient-based localization [C]// 2017 IEEE International Conference on Computer Vision (ICCV). Piscataway: IEEE. 2017: 618-626. [50] Chattopadhay A, Sarkar A, Howlader P, et al. Grad-cam++: Generalized gradient-based visual explanations for deep convolutional networks [C] //2018 IEEE Winter Conference on Applications of Computer Vision (WACV). Piscataway: IEEE. 2018: 839-847,. [51] Armanious K, Abdulatif S, Shi W, et al. Age-Net: an MRI-basediterative framework for brain biological age estimation [J]. IEEE Trans Med Imaging, 2021, 40(7): 1778-1791. [52] 纪守领, 李进锋, 杜天宇,等. 机器学习模型可解释性方法、应用与安全研究综述[J].计算机研究与发展,2019,56(10):2071-2096. [53] Landecker W, Thomure MD, Bettencourt LM, et al. Interpreting individual classifications of hierarchical networks [C]// 2013 IEEE Symposium on Computational Intelligence and DataMining (CIDM). Piscataway: IEEE. 2013: 32-38. [54] Bach S, Binder A, Montavon G, et al. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation[J]. PLoS ONE, 2015, 10(7): e0130140. [55] Yuan Hao, Yu Haiyang, Gui Shurui, et al.Explainability in graph neural networks: A taxonomic survey [EB/OL]. https://arxiv.org/pdf/ 2012.15445.pdf, 2020-11-08/2022-01-05. [56] Lee G, Fujita H. Deep Learning in Medical Image Analysis Challenges and Applications Preface[J]. Deep Learning in Medical Image Analysis: Challenges and Applications, 2020, 1213: V-VI. [57] Yang C, Rangarajan A, Ranka S. Visual explanations from deep 3D convolutional neural networks for Alzheimer′s disease classification [J]. AMIA Annu Symp Proc, 2018, 2018:1571-1580. [58] Eitel F, Schulz MA, Seiler M, et al. Promises and pitfalls of deep neural networks in neuroimaging-based psychiatric research[J]. Experimental Neurology, 2021, 339: 113608. [59] Eitel F, Ritter K. Testing the robustness of attribution methods for convolutional neural networks in MRI-based Alzheimer′s disease classification [M]// Interpretability of Machine Intelligence in Medical Image Computing and Multimodal Learning for Clinical Decision Support. Shenzhen: Springer, 2019: 3-11. [60] Dyrba M, Hanzig M, Altenstein S, et al. Improving 3D convolutional neural network comprehensibility via interactive visualization of relevance maps: evaluation in Alzheimer′s disease [J]. Alzheimers Res Ther, 2021, 13(1): 1-18.
