Research on Multi Network Carotid Artery Image Classification and Detection Based on TransferLearning and CNN
Sui Xiaoyu1, Han Jing2, Cao Yankun1, Mi Jia3, Song Yanyun4, Wang Jianlei5, Wang Chun5*, Liu Zhi1*
1(School of Information Science and Engineering, Shandong University, Qingdao 266237, Shandong, China) 2(School of Information and Control Engineering, China University of Mining and Technology, Xuzhou 221116, Jiangsu,China) 3(Special Examination Department,Shandong Provincial Third Hospital, Jinan 250031, China) 4(Shandong Sports Rehabilitation Research Center, Jinan 250102, China) 5(Center for Optics Research and Engineering, Shandong University, Qingdao 266237, Shandong, China)
Abstract：Carotid ultrasound is a main and convenient method for plaque diagnosis. Therefore, it is very important to obtain accurate information about plaque from ultrasound images for further clinical diagnosis. Due to the noise interference of ultrasonic machine and the difference of manual technical operation, the displayed section image is not clear and standard, which is easy to lead to false detection or missed detection of the plaque. In this work, a deep learning algorithm based on migration learning and CNN was proposed to realize the research of more accurate identification of carotid plaque. Firstly, 2591 longitudinal ultrasound images with clear and fuzzy carotid artery were selected to classify and control the lumen quality through ResNet network; After the lumen classification, 1114 longitudinal images with clear soft and hard plaque information were selected. The carotid lumen and plaque were classified and detected by RetinaNet network based on migration learning, and the comparative experiment was carried out by using Faster R-CNN and SSD network. For the lumen classification network, the classification accuracy of the test set was 93%. For the lumen and plaque classification detection network, 113 test set images were used to obtain the average accuracy of lumen detection, which reached 1 when the intersection union ratio (IOU) value was 0.5, 0.988 when the IOU value was 0.75, 0.838 when the IOU value was 0.5: 0.95, and the average recall rate reached 0.869, which were higher than those of Faster R-CNN and SSD networks; The average accuracy of hard plaque and soft plaque detection was 0.899 and 0.855 when IOU = 0.5, and the average recall was 0.609 and 0.578 respectively. Before the classification and recognition of carotid plaque, the quality classification control of carotid lumen image can effectively avoid the false detection of plaque caused by unclear image, improve the correctness of plaque detection, and is of great significance for the follow-up three-dimensional reconstruction of carotid artery.
隋小瑜, 韩晶, 曹艳坤, 米加, 宋延云, 王剑磊, 王春, 刘治. 基于迁移学习和卷积神经网络多网络颈动脉图像分类检测研究[J]. 中国生物医学工程学报, 2024, 43(3): 267-277.
Sui Xiaoyu, Han Jing, Cao Yankun, Mi Jia, Song Yanyun, Wang Jianlei, Wang Chun, Liu Zhi. Research on Multi Network Carotid Artery Image Classification and Detection Based on TransferLearning and CNN. Chinese Journal of Biomedical Engineering, 2024, 43(3): 267-277.
[1] Grau AJ, Weimar C, Buggle F, et al. Risk factors, outcome, and treatment in subtypes of ischemic stroke: the german stroke data bank[J]. Stroke, 2001, 32(11):2559-2566. [2] Mayor I, Momjian S, Lalive P, et al. Carotid plaque: comparison between visual and grey-scale median analysis[J]. Ultrasound in Medicine & Biology, 2003, 29(7):961-966. [3] Guang Y, He W, Ning B, et al. Deep learning-based carotid plaque vulnerability classification with multicentre contrast-enhanced ultrasound video: a comparative diagnostic study[J]. BMJ Open. 2021,11(8): e047528. [4] Hennerici M, Meairs S. Imaging arterial wall disease[J]. Cerebrovascular Diseases, 2000, 10(Suppl 5):9-20. [5] 彭莉玲. 颈动脉斑块的影像学评估现状和展望[J]. 功能与分子医学影像学:电子版, 2017, 6(4):1342-1348. [6] Gillies RJ, Kinahan PE, Hricak H. Radiomics: images are more than pictures, they are data[J]. Radiology, 2016, 278(2):563-577. [7] Hinton G E, Salakhutdinov RR. Reducing the dimensionality of data with neural networks[J]. Science, 2006,313(5786):504-507. [8] Menchón-Lara RM, Sancho-Gómez JL, Bueno-Crespo A. Early-stage atherosclerosis detection using deep learning over carotid ultrasound images[J]. Applied Soft Computing, 2016,49:616-628. [9] 赵媛,孙夏,Aaron Fenster,等.一种基于深度学习的颈动脉斑块超声图像识别方法[J].中国医疗器械信息,2017,23(9):9-11,24. [10] Alom MZ, Taha TM, Yakopcic C, et al. The history began from alexnet: a comprehensive survey on deep learning approaches [EB/OL].https://arxiv.org/abs/1803.01164v1, 2018-08-12/2022-03-13. [11] Redmon J, Divvala S, Girshick R, et al. You only look once: Unified, real-time object detection[C]// 2016 IEEE conference on computer vision and pattern recognition. Las Vegas: IEEE,2016,779-788. [12] Lin TY, Goyal P, Girshick R, et al. Focal loss for dense object detection[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2020, 42(2):318-327. [13] Liu W, Anguelov D, Erhan D, et al. Ssd: Single shot multibox detector[C]// European Conference on Computer Vision. Cham: Springer, 2016: 21-37. [14] Yang M, Xiao X, Liu Z, et al. Deep RetinaNet for dynamic left ventricle detection in multiview echocardiography classification[J]. Scientific Programming, 2020, 2020(5):1-6. [15] Ren S, He K, Girshick R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017, 39(6):1137-1149. [16] 张军连.颈动脉粥样硬化斑块超声特征对脑梗塞的预测价值分析[J].影像研究与医学应用,2020,4(24):132-134. [17] 杨德斌,金琳,王迎春.超声评价颈动脉粥样硬化斑块稳定性的研究进展[J].中国医学影像学杂志,2016,24(9):717-720. [18] He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE,2016, 770-778. [19] 美通社.”e健康中国心”项目在全国推行,打造健康扶贫新模式[EB/OL].https://baijiahao.baidu.com/s?id=1639836197382837204&wfr=spider&for=pc, 2019-07-23/2022-03-13. [20] Lin TY, Dollar P, Girshick R, et al. Feature pyramid networks for object detection[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 2117-2125. [21] Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 39(4):640-651. [22] Li X, Chang D, Tian T, et al. Large-margin regularized softmax cross-entropy loss[J]. IEEE Access, 2019, 7:19572-19578. [23] Kang K, Pang G, Zhao X, et al. A new benchmark for instance-level image classification[J]. IEEE Access, 2020, 8:70306-70315. [24] Sokolova M, Japkowicz N, Szpakowicz S. Beyond accuracy, F-score and ROC: a family of discriminant measures for performance evaluation[C]// Australasian JointConference on Artificial Intelligence. Berlin: Springer, 2006:1015-1021. [25] Cen LP, Ji J, Lin JW, et al. Automatic detection of 39 fundus diseases and conditions in retinal photographs using deep neural networks[J]. Nature Communications, 2021,12(1):1-13. [26] 何马均.深度学习框架TensorFlow的高效分布式并行算法研究[D].成都: 电子科技大学,2019. [27] Pan SJ, Qiang Y. A survey on transfer learning[J]. IEEE Transactions on Knowledge and Data Engineering, 2010, 22(10):1345-1359. [28] Krizhevsky A, Sutskever I, Hinton G. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6):84-90. [29] Zhou Y, Dong F, Liu Y, et al. Forecasting emerging technologies using data augmentation and deep learning[J]. Scientometrics, Springer Akadémiai Kiadó, 2020, 123(1):1-29. [30] Morid MA, Borjali A, Fiol GD. A scoping review of transfer learning research on medical image analysis using ImageNet[J]. Computers in Biology and Medicine, 2021, 128:104115. [31] 蔡梦媛,周然,程新耀,等.基于深度学习的颈动脉超声图像斑块分割算法[J].生命科学仪器,2020,18(3):45-53. [32] 沈冲冲,周小安,安相静,等.深度学习算法在颈动脉超声图像斑块分割中的应用研究[J].智能计算机与应用,2021,11(1):84-88.
