Lumen and Media-Adventitia Borders Detection in IVUS Images Using Conditional Generative Adversarial Networks
Yuan Shaofeng1,2, Yang Feng1,2*, Xu Lin3 *, Wu Yangyang1,2, Huang Jing1,2, Liu Yaqin1,2
1.(School of Biomedical Engineering, Southern Medical University, Guangzhou 510515, China) 2.(Guangdong Provincial Key Laboratory of Medical Image Processing, Southern Medical University, Guangzhou 510515, China); 3.(Department of Cardiology, Southern Theater General Hospital (Guangzhou General Hospital of Guangzhou Military Region), PLA, Guangzhou 510010, China)
Abstract This paper presented an improved method based on stacked hourglass networks (SHGN) and conditional generative adversarial networks (C-GAN) for detecting lumen (LU) and media-adventitia (MA) borders in intravascular ultrasound (IVUS) images to overcome the problems of interference factors, such as different kinds of plaques, ultrasound shadow and branches of vessels. According to the specific shape of coronary artery, several approaches used for data augmentation were applied to increase the number of IVUS samples in training set, including rotating every other 10 degrees, scaling up or down and implementing gamma transformation, in order to reduce the risk of over-fitting during the training stage. Subsequently, in the spirit of adversarial training, a joint loss function based on L1 or L2 loss was constructed and the networks learned a mapping between input IVUS images and their corresponding ground-truths, to segment the whole image into three regions: non-vessel, plaque and lumen regions. Lastly, LU and MA borders were obtained by a thresholding method. Experiments were performed on an international reference dataset containing 435 IVUS images from 10 patients and a small dataset of 100 frames from department of cardiology in Guangzhou General Hospital of Guangzhou Military Region. Quantitative evaluation results showed that the average Jaccard Measures (JM) of two borders of interest were 93% and 95%, the average percentage of area difference measures (PAD) for both was 3%, and average Hausdorff distance measures (HD) were 0.19 mm and 0.16 mm, respectively. These evaluation metrics could meet clinical demands, and the performance of our method was better than those of algorithms evaluated on the same reference dataset and the Pix2Pix model′s performance. Successful detection results on Guangzhou General Hospital dataset demonstrated that the proposed method has good generalization on cross-dataset.
[1] 刘茜蒨, 刘健, 陈芸. 轻松掌握血管内超声 [M]. 北京: 人民军医出版社, 2009: 51-94. [2] Katouzian A, Angelini E, Carlier S, et al. A state-of-the-art review on segmentation algorithms in intravascular ultrasound (IVUS) images [J]. IEEE Transactions on Information Technology in Biomedicine, 2012, 16(5): 823-831. [3] 邢东, 杨丰, 黄靖, 等. 结合硬斑块特征的心血管内超声图像中-外膜边缘检测 [J]. 中国生物医学工程学报, 2012, 31(1): 25-31. [4] Giannoglou GD, Chatzizisis YS, Koutkias V, et al. A novel active contour model for fully automated segmentation of intravascular ultrasound images: In vivo validation in human coronary arteries [J]. Computers in Biology and Medicine, 2007, 39(7): 1292-1302. [5] Essa E, Xie X, Sazonov I, et al. Shape prior model for Media-Adventitia border segmentation in IVUS using graph cut [C] // Medical Image Computing and Computer-Assisted Intervention Workshop on Medical Computer Vision. Nice: Springer Berlin Heidelberg, 2012: 114-123. [6] Unal G, Bucher S, Carlier S, et al. Shape-driven segmentation of the arterial wall in intravascular ultrasound images [J]. IEEE Transactions on Information Technology in Biomedicine, 2008, 12(3): 335-374. [7] Mendizabal-Ruiz EG, Rivera M, Kakadiaris IA. Segmentation of the luminal border in intravascular ultrasound B-mode images using a probabilistic approach [J]. Medical Image Analysis, 2013, 17(6): 649-670. [8] Rotger D, Radeva P, Fernandez-Nofrerias E, et al. Blood detection in IVUS images for 3D volume of lumen changes measurement due to different drugs administration [C] // International Conference on Computer Analysis of Images and Patterns. Vienna: Springer Berlin Heidelberg, 2007: 285-292. [9] 林慕丹, 杨丰, 梁淑君, 等. 结合先验形状信息和序贯学习的血管内超外弹力膜检测 [J]. 中国图象图形学报, 2016, 21(5): 646-656. [10] Su S, Hu Z, Lin Q, et al. An artificial neural network method for lumen and media-adventitia border detection in IVUS [J]. Computerized Medical Imaging and Graphics, 2017, 57: 29-39. [11] Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks [C] // Advances in Neural Information Processing Systems. Lake Tahoe: NIPS Foundation, 2012: 1097-1105. [12] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition [J]. arXiv:1409.1556, 2014. [13] Szegedy C, Liu W, Jia Y, et al. Going deeper with convolution [C] // International Conference on Computer Vision and Pattern Recognition. Boston: IEEE, 2015: 1-9. [14] He K, Zhang X, Ren, S, et al. Deep residual learning for image recognition [C] // International Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778. [15] Shelhamer E, Long J, Darrell T. Fully convolutional networks for semantic segmentation [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4): 640-651. [16] 蔡程飞, 徐军, 梁莉, 等. 基于深度卷积网络的结直肠全扫描病理图像的多种组织分割 [J]. 中国生物医学工程学报, 2017, 36(5): 632-636. [17] 余镇, 吴凌云, 倪东, 等. 基于深度学习的胎儿颜面部超声标准切面自动识别 [J]. 中国生物医学工程学报, 2017, 36(3): 267-275. [18] Litjens G, Kooi T, Bejnordi BE, et al. A survey on deep learning in medical image analysis [J]. Medical Image Analysis, 2017, 42: 60-88. [19] Newell A, Yang K, Deng J. Stacked hourglass networks for human pose estimation [C] // European Conference on Computer Vision. Amsterdam: Springer International Publishing, 2016: 483-499. [20] Tajbakhsh N, Shin JY, Gurudu SR, et al. Convolutional neural networks for medical image analysis: full training or fine tuning? [J]. IEEE Transactions on Medical Imaging, 2016, 35(5): 1299-1312. [21] Goodfelow IJ, Pouget-Abadie J, Mirza M, et al. Generative adversarial nets [C] // Advances in Neural Information Processing Systems. Montreal: NIPS Foundation, 2014: 2672-2680. [22] Mirza M, Osindero S. Conditional generative adversarial nets [J]. ArXiv: 1411.1784, 2014. [23] Luc P, Couprie C, Chintala S, et al. Semantic segmentation using adversarial networks [J]. arXiv: 1611.08408,2016. [24] Balocco S, Gatta C, Ciompi F, et al. Standardized evaluation methodology and reference database for evaluating IVUS image segmentation [J]. Computerized Medical Imaging and Graphics, 2014, 38(2): 70-90. [25] Isola P, Zhu JY, Zhou T, et al. Image-to-image translation with conditional adversarial networks [C] // International Conference on Computer Vision and Pattern Recognition. Hawaii: IEEE, 2017:1125-1134. [26] Shen X, Hertzmann A, Jia J, et al. Automatic portrait segmentation for image stylization [J]. Computer Graphics Forum, 2016, 35(2): 93-102. [27] Kingma DP, Welling M. Auto-encoding variational bayes [J]. arXiv: 1312.6114, 2013. [28] Radford A, Metz L, Chintala S. Unsupervised representation learning with deep convolutional generative adversarial networks [J] arXiv:1511.06434, 2016. [29] Pathak D, Krahenhuhl P, Donahue J, et al. Context encoders: feature learning by inpainting [C] // International Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 2536-2544. [30] Hinton G, Salakhutdinov R. Reducing the dimensionality of data with neural networks [J]. Science, 2006, 313(5786): 504-507. [31] Fu J, Liu J, Wang Y, Lu H. Stacked deconvolutional network for semantic segmentation [J]. Arxiv: 1708.04943, 2017. [32] Liu Y, Nguyen DM, Deligiannis N, et al. Hourglass-shapenetwork based semantic segmentation for high resolution aerial imagery [J]. Remote Sens, 2017, 9(6): 522. [33] Vigneault DM, Xie W, Ho CY, et al. W-Net: Fully automatic, multi-view cardiac mr detection, orientation, and segmentation with deep neural networks [J]. Arxiv: 1711.01094, 2017. [34] Wu L, Yang X, Li S, et al. Cascaded fully convolutional networks for automatic prenatal ultrasound image segmentation [C] // International Symposium on Biomedical Imaging. Melbourne: IEEE, 2017: 663-666. [35] Ronneberger O, Fisher P, Brox T. U-Net: convolutional networks for biomedical image segmentation [C] // Medical Image Computing and Computer-Assisted Intervention. Munich: Springer Berlin Heidelberg, 2015: 234-241. [36] Abadi M, Agarwal A, Barham P, et al. TensorFlow: Large-scale machine learning on heterogeneous distributed systems [J]. arXiv: 1606.04467, 2016. [37] Kingma DP, Ba JL. Adam: A method for stochastic optimization [J]. arXiv:1412.6980, 2015. [38] Zhang Q, Wang Y, Wang, et al. Automatic segmentation of calcifications in intravascular ultrasound images using snakes and contourlet transform [J]. Ultrasound in Medicine and Biology, 2010, 36(1): 111-129. [39] Athanasiou LS, Karvelis PS, Tsakanikas VD, et al. A novel semiautomated atherosclerotic plaque characterization method using grayscale intravascular ultrasound images: comparison with virtual histology [J]. IEEE Transactions on Information Technology in Biomedicine, 2012, 16(3): 391-400. [40] Taki A, Hetterich H, Roodaki A, et al. A new approach for improving coronary plaque component analysis based on intravascular ultrasound images [J]. Ultrasound in Medicine and Biology, 2010, 36(8): 1245-1258.
