Automatic Classification of Retinal Optical Coherence Tomography Images via Convolutional Neural Networks with Joint Decision
Wang Chong1, He Xingxin1, Fang Leyuan1*, Guo Siyu1, Chen Xiangdong2, Nie Fujiao2
1College of Electrical and Information Engineering, Hunan University, Changsha 410082, China; 2Department of Ophthalmology, The First Affiliated Hospital of Hunan University of Chinese Medicine, Changsha 410007, China
Abstract Optical coherence tomography (OCT) can provide in vivo three dimensional (3D) cross-sectional imaging of human retina at micrometer resolutions, which is a significant tool for the diagnosis and the progression tracking of retinal diseases. In the clinical diagnosis, ophthalmologists need to manually identify various macular lesions at each cross section of the 3D OCT images. Such manual analysis is time-consuming and often yields subjective results. Therefore, it is urgent to develop an automatic classification algorithm to improve the efficiency of OCT-based analysis in daily clinical practice. This paper proposed a novel automatic method, based on the convolutional neural networks (CNN) with joint decision for the classification of OCT images. The proposed joint-decision CNN-based method was able to automatically learn multiple-layer features from original OCT images through a convolutional neural network, these features in each layer were simultaneously utilized to separately make decision of classification. Finally, these decisions were fused together to achieve the final classification. The experimental results on Duke data sets (3231 B-scans) showed that the proposed joint-decision CNN-based method achieved average accuracy of 94.5%, average sensitivity 90.5% and average specificity 95.8%, for the automatic identification of normal macula, age-related macular degeneration and macular edema respectively. The experimental results on HUCM data sets (4322 B-scans) showed that the proposed joint-decision CNN-based method achievesd average accuracy of 89.6%, average sensitivity 88.8% and average specificity 90.8%. The results proved that the richly multiple-layer features of CNN could be used to accurately classify retinal OCT images, hence the algorithm provided effective technical support for the aided diagnosis of retinal diseases in clinical practice.
Wang Chong,He Xingxin,Fang Leyuan, et al. Automatic Classification of Retinal Optical Coherence Tomography Images via Convolutional Neural Networks with Joint Decision[J]. Chinese Journal of Biomedical Engineering, 2018, 37(6): 641-648.
[1] 刘祖国. 眼科学基础 [M]. 北京: 人民卫生出版社, 2004: 36-78. [2] Quillen DA. Common causes of vision loss in elderly patients [J]. American Family Physician, 1999, 60(1): 99-108. [3] Mehta S. Age-related macular degeneration [J]. Primary Care: Clinics in Office Practice, 2015, 42(3): 377-391. [4] Abdelsalam A, Delpriore L, Zarbin MA. Drusen in age-related macular degeneration: pathogenesis, natural course, and laser photocoagulation-induced regression [J]. Survey of Ophthalmology, 1999, 44(1): 1-29. [5] Antonetti DA, Barber AJ, Bronson SK, et al. Diabetic retinopathy [J]. Diabetes, 2006, 55(9): 2401-2411. [6] Hirai FE, Knudtson MD, Klein BE, et al. Clinically significant macular edema and survival in type 1 and type 2 diabetes [J]. American Journal of Ophthalmology, 2008, 145(4): 700-706. [7] Huang D, Swanson EA, Lin CP, et al. Optical coherence tomography [J]. Science, 1991, 254(5035): 1178-1181. [8] Puliafito CA, Hee MR, Lin CP, et al. Imaging of macular diseases with optical coherence tomography [J]. Ophthalmology, 1995, 102(2): 217-229. [9] Drexler W, Fujimoto JG. State-of-the-art retinal optical coherence tomography [J]. Progress in Retinal and Eye Research, 2008, 27(1): 45-88. [10] Tomlins PH, Wang RK. Theory, developments and applications of optical coherence tomography [J]. Journal of Physics D: Applied Physics, 2005, 38(15): 2519-2535. [11] Liu Yuying, Chen Mei, Ishikawa H, et al. Automated macular pathology diagnosis in retinal OCT images using multi-scale spatial pyramid and local binary patterns in texture and shape encoding [J]. Medical Image Analysis, 2011, 15(5): 748-759. [12] Srinivasan PP, Kim LA, Mettu PS, et al. Fully automated detection of diabetic macular edema and dry age-related macular degeneration from optical coherence tomography images [J]. Biomedical Optics Express, 2014, 5(10): 3568-3577. [13] Lemaître G, Rastgoo M, Massich J, et al. Classification of SD-OCT volumes using local binary patterns: Experimental validation for DME detection [J]. Journal of Ophthalmology, 2016, 4(9): 28-54. [14] Wang Yu, Zhang Yaonan, Yao Zhaomin, et al. Machine learning based detection of age-related macular degeneration (AMD) and diabetic macular edema (DME) from optical coherence tomography (OCT) images [J]. Biomedical Optics Express, 2016, 7(12): 4928-4940. [15] Fang Leyuan, Wang Chong, Li Shutao, et al. Automatic classification of retinal three-dimensional optical coherence tomography images using principal component analysis network with composite kernels [J]. Journal of Biomedical Optics, 2017, 22(11): 116-126. [16] Karri S, Chakraborty D, Chatterjee J. Transfer learning based classification of optical coherence tomography images with diabetic macular edema and dry age-related macular degeneration [J]. Biomedical Optics Express, 2017, 8(2): 579-592. [17] Sun Yankui, Li Shan, Sun Zhongyang. Fully automated macular pathology detection in retina optical coherence tomography images using sparse coding and dictionary learning [J]. Journal of Biomedical Optics, 2017, 22(1): 11-23. [18] LeCun Y, Bengio Y, Hinton G. Deep learning [J]. Nature, 2015, 521(7553): 436-444. [19] Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks [C] //International Conference on Neural Information Processing Systems. Nevada: Curran Associates Inc, 2012: 1097-1105. [20] Szegedy C, Liu Wei, Jia Yangqing, et al. Going deeper with convolutions [C] //Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Boston: IEEE, 2015: 884-897. [21] Litjens G, Kooi T, Bejnordi BE, et al. A survey on deep learning in medical image analysis [J]. Medical Image Analysis, 2017, 42(8): 60-88. [22] 余镇, 吴凌云, 倪东, 等. 基于深度学习的胎儿颜面部超声标准切面自动识别 [J]. 中国生物医学工程学报, 2017, 36(3): 267-275. [23] 王金甲, 刘青玉, 陈浩. 基于深度卷积神经网络的帕金森步态识别 [J]. 中国生物医学工程学报, 2017, 36(4): 418-425. [24] 周超, 徐军, 罗波. 基于深度卷积网络和结合策略的乳腺组织病理图像细胞核异型性自动评分 [J]. 中国生物医学工程学报, 2017, 36(3): 276-283. [25] 刘明, 李国军, 郝华青,等. 基于卷积神经网络的T波形态分类 [J]. 自动化学报, 2016, 42(9): 1339-1346. [26] Ren S, Girshick R, Girshick R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149. [27] Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 39(4): 640-651. [28] LeCun Y, Boser BE, Denker JS, et al. Handwritten digit recognition with a back-propagation network [C] //International Conference on Neural Information Processing Systems. Denver: Curran Associates Inc, 1990: 396-404. [29] Schmidhuber J. Deep learning in neural networks: An overview [J]. Neural Networks, 2015, 61(6): 85-117. [30] Qin Hongwei, Yan Junjie, Li Xiu, et al. Joint training of cascaded CNN for face detection [C] //Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2016: 3456-3465. [31] Xie Yutong, Zhang Jianpeng, Xia Yong, et al. Fusing texture, shape and deep model-learned information at decision level for automated classification of lung nodules on chest CT [J]. Information Fusion, 2018, 42(3): 102-110. [32] Abadi M, Barham P, Chen Jianmin, et al. TensorFlow: A system for large-scale machine learning [J]. Computer Science, 2016, 16(3): 265-283. [33] Kingma DP, Ba J. Adam: A method for stochastic optimization [J]. Computer Science, 2014, 62(6): 660-681.
