Automatic Classification of Retinal Optical Coherence Tomography Images via Convolutional Neural Networks with Joint Decision
Wang Chong1, He Xingxin1, Fang Leyuan1*, Guo Siyu1, Chen Xiangdong2, Nie Fujiao2
1College of Electrical and Information Engineering, Hunan University, Changsha 410082, China; 2Department of Ophthalmology, The First Affiliated Hospital of Hunan University of Chinese Medicine, Changsha 410007, China
摘要光学相干断层扫描(OCT)技术能实现视网膜的高分辨率三维层析成像,对视网膜疾病类型的诊断和发展阶段的分析具有至关重要的作用。临床基于 OCT 图像的视网膜疾病诊断主要依靠眼科医生对图像中病变结构的分析,这一人工分析过程不仅耗时而且易产生主观的误判。研究视网膜疾病的自动分析和诊断技术将极大减轻眼科医生的工作量,是实现高效诊疗的有效途径。针对视网膜OCT图像自动分类,构建一种联合决策的卷积神经网络分类模型。该模型利用卷积神经网络从原始输入OCT图像中自动地学习不同层级的特征,同时在网络多个卷积层上设计多个决策层,这些决策层能够根据网络中不同尺度的特征图分别对OCT图像分类,最后模型融合所有决策层的分类结果做出最终决策。在Duke数据集(3 231张OCT图像)上的实验结果表明,基于多层级特征联合决策的卷积神经网络分类模型对正常视网膜、视网膜年龄相关性黄斑变性和视网膜黄斑水肿的平均识别准确率达到94.5%,灵敏性达到90.5%,特异性达到95.8%。在HUCM数据集(4 322张OCT图像)上的实验结果表明,基于多层级特征联合决策的卷积神经网络分类模型的平均识别准确率达到89.6%,灵敏性达到88.8%,特异性达到90.8%。充分利用卷积神经网络中丰富的多层级特征,能够有效地对视网膜OCT图像实现准确的分类,为临床上视网膜疾病的辅助诊断提供技术支撑。
Abstract：Optical coherence tomography (OCT) can provide in vivo three dimensional (3D) cross-sectional imaging of human retina at micrometer resolutions, which is a significant tool for the diagnosis and the progression tracking of retinal diseases. In the clinical diagnosis, ophthalmologists need to manually identify various macular lesions at each cross section of the 3D OCT images. Such manual analysis is time-consuming and often yields subjective results. Therefore, it is urgent to develop an automatic classification algorithm to improve the efficiency of OCT-based analysis in daily clinical practice. This paper proposed a novel automatic method, based on the convolutional neural networks (CNN) with joint decision for the classification of OCT images. The proposed joint-decision CNN-based method was able to automatically learn multiple-layer features from original OCT images through a convolutional neural network, these features in each layer were simultaneously utilized to separately make decision of classification. Finally, these decisions were fused together to achieve the final classification. The experimental results on Duke data sets (3231 B-scans) showed that the proposed joint-decision CNN-based method achieved average accuracy of 94.5%, average sensitivity 90.5% and average specificity 95.8%, for the automatic identification of normal macula, age-related macular degeneration and macular edema respectively. The experimental results on HUCM data sets (4322 B-scans) showed that the proposed joint-decision CNN-based method achievesd average accuracy of 89.6%, average sensitivity 88.8% and average specificity 90.8%. The results proved that the richly multiple-layer features of CNN could be used to accurately classify retinal OCT images, hence the algorithm provided effective technical support for the aided diagnosis of retinal diseases in clinical practice.
[1] 刘祖国. 眼科学基础 [M]. 北京: 人民卫生出版社, 2004: 36-78. [2] Quillen DA. Common causes of vision loss in elderly patients [J]. American Family Physician, 1999, 60(1): 99-108. [3] Mehta S. Age-related macular degeneration [J]. Primary Care: Clinics in Office Practice, 2015, 42(3): 377-391. [4] Abdelsalam A, Delpriore L, Zarbin MA. Drusen in age-related macular degeneration: pathogenesis, natural course, and laser photocoagulation-induced regression [J]. Survey of Ophthalmology, 1999, 44(1): 1-29. [5] Antonetti DA, Barber AJ, Bronson SK, et al. Diabetic retinopathy [J]. Diabetes, 2006, 55(9): 2401-2411. [6] Hirai FE, Knudtson MD, Klein BE, et al. Clinically significant macular edema and survival in type 1 and type 2 diabetes [J]. American Journal of Ophthalmology, 2008, 145(4): 700-706. [7] Huang D, Swanson EA, Lin CP, et al. Optical coherence tomography [J]. Science, 1991, 254(5035): 1178-1181. [8] Puliafito CA, Hee MR, Lin CP, et al. Imaging of macular diseases with optical coherence tomography [J]. Ophthalmology, 1995, 102(2): 217-229. [9] Drexler W, Fujimoto JG. State-of-the-art retinal optical coherence tomography [J]. Progress in Retinal and Eye Research, 2008, 27(1): 45-88. [10] Tomlins PH, Wang RK. Theory, developments and applications of optical coherence tomography [J]. Journal of Physics D: Applied Physics, 2005, 38(15): 2519-2535. [11] Liu Yuying, Chen Mei, Ishikawa H, et al. Automated macular pathology diagnosis in retinal OCT images using multi-scale spatial pyramid and local binary patterns in texture and shape encoding [J]. Medical Image Analysis, 2011, 15(5): 748-759. [12] Srinivasan PP, Kim LA, Mettu PS, et al. Fully automated detection of diabetic macular edema and dry age-related macular degeneration from optical coherence tomography images [J]. Biomedical Optics Express, 2014, 5(10): 3568-3577. [13] Lemaître G, Rastgoo M, Massich J, et al. Classification of SD-OCT volumes using local binary patterns: Experimental validation for DME detection [J]. Journal of Ophthalmology, 2016, 4(9): 28-54. [14] Wang Yu, Zhang Yaonan, Yao Zhaomin, et al. Machine learning based detection of age-related macular degeneration (AMD) and diabetic macular edema (DME) from optical coherence tomography (OCT) images [J]. Biomedical Optics Express, 2016, 7(12): 4928-4940. [15] Fang Leyuan, Wang Chong, Li Shutao, et al. Automatic classification of retinal three-dimensional optical coherence tomography images using principal component analysis network with composite kernels [J]. Journal of Biomedical Optics, 2017, 22(11): 116-126. [16] Karri S, Chakraborty D, Chatterjee J. Transfer learning based classification of optical coherence tomography images with diabetic macular edema and dry age-related macular degeneration [J]. Biomedical Optics Express, 2017, 8(2): 579-592. [17] Sun Yankui, Li Shan, Sun Zhongyang. Fully automated macular pathology detection in retina optical coherence tomography images using sparse coding and dictionary learning [J]. Journal of Biomedical Optics, 2017, 22(1): 11-23. [18] LeCun Y, Bengio Y, Hinton G. Deep learning [J]. Nature, 2015, 521(7553): 436-444. [19] Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks [C] //International Conference on Neural Information Processing Systems. Nevada: Curran Associates Inc, 2012: 1097-1105. [20] Szegedy C, Liu Wei, Jia Yangqing, et al. Going deeper with convolutions [C] //Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Boston: IEEE, 2015: 884-897. [21] Litjens G, Kooi T, Bejnordi BE, et al. A survey on deep learning in medical image analysis [J]. Medical Image Analysis, 2017, 42(8): 60-88. [22] 余镇, 吴凌云, 倪东, 等. 基于深度学习的胎儿颜面部超声标准切面自动识别 [J]. 中国生物医学工程学报, 2017, 36(3): 267-275. [23] 王金甲, 刘青玉, 陈浩. 基于深度卷积神经网络的帕金森步态识别 [J]. 中国生物医学工程学报, 2017, 36(4): 418-425. [24] 周超, 徐军, 罗波. 基于深度卷积网络和结合策略的乳腺组织病理图像细胞核异型性自动评分 [J]. 中国生物医学工程学报, 2017, 36(3): 276-283. [25] 刘明, 李国军, 郝华青,等. 基于卷积神经网络的T波形态分类 [J]. 自动化学报, 2016, 42(9): 1339-1346. [26] Ren S, Girshick R, Girshick R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149. [27] Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 39(4): 640-651. [28] LeCun Y, Boser BE, Denker JS, et al. Handwritten digit recognition with a back-propagation network [C] //International Conference on Neural Information Processing Systems. Denver: Curran Associates Inc, 1990: 396-404. [29] Schmidhuber J. Deep learning in neural networks: An overview [J]. Neural Networks, 2015, 61(6): 85-117. [30] Qin Hongwei, Yan Junjie, Li Xiu, et al. Joint training of cascaded CNN for face detection [C] //Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2016: 3456-3465. [31] Xie Yutong, Zhang Jianpeng, Xia Yong, et al. Fusing texture, shape and deep model-learned information at decision level for automated classification of lung nodules on chest CT [J]. Information Fusion, 2018, 42(3): 102-110. [32] Abadi M, Barham P, Chen Jianmin, et al. TensorFlow: A system for large-scale machine learning [J]. Computer Science, 2016, 16(3): 265-283. [33] Kingma DP, Ba J. Adam: A method for stochastic optimization [J]. Computer Science, 2014, 62(6): 660-681.
