Glioma Segmentation Based on Feature Selection of Multi-Modal MR Images
Cheng Juan1#, Zhang Chuya1, Liu Yu1#*, Li Chang1, Zhu Zhiqin2, Chen Xun3#
1(Department of Biomedical Engineering, Hefei University of Technology, Hefei 230009, China) 2(College of Automation, Chongqing University of Posts and Telecommunications, Chongqing 400065, China) 3(Department of Electronic Engineering and Information Science, University of Science and Technology of China, Hefei 230026, China)
Abstract：Glioma segmentation based on multi-modal MR images plays a positive role for the diagnosis and treatment of tumors. It is known that different modalities of MR images can provide different properties of information representing pathological tissues. Currently, an increasing number of deep-learning-based glioma segmentation methods have been proposed to segment brain gliomas utilizing multi-modal MR images. However, these methods usually stack the original features derived from multi-modal MR images channel by channel, and roughly take the stacked features as inputs, leading to an inadequate feature mining and an unsatisfactory segmentation performance. To solve this problem, this paper proposed to segment three glioma regions with a two-stage segmentation scheme, with each stage having a feature selection module and a segmentation network. The first stage of the segmentation aimed to segment peripheral edema regions, while the second stage tried to segment necrosis/non-enhancing tumor and enhancing tumor regions. Besides, the first-stage segmentation results provided essential location information that would benefit for segmenting the other two tumor regions during the second stage. For each stage, a multi-modal feature selection module was designed to automatically extract effective and cross-modal-fused features from each modality of MR images, and then these features were sent to each following segmentation network. The segmentation network was composed of a V-Net and a variational autoencoder (VAE). Experiments were conducted on three public brain tumor datasets including BraTS2018, BraTS2019 and BraTS2020. Specifically, for dataset BraTS2018, the average Dice scores of the proposed method for segmenting the whole tumor (WT), the tumor core (TC), and the enhanced tumor (ET) regions reached 0.898, 0.854, and 0.818, respectively, while the Hausdorff95 distance of the proposed method for segmenting the aforementioned three regions reached 4.072, 6.179, and 3.763, respectively. As for dataset BraTS2019, the average Dice scores of the proposed method for segmenting the abovementioned three tumor regions reached 0.892, 0.839, and 0.800, respectively, while the corresponding Hausdorff95 distance of the proposed method can reach to 6.168, 7.077 and 3.807, respectively. As for dataset BraTS2020, the average Dice scores of the proposed method for segmenting the same three regions reached 0.896, 0.837, and 0.803, respectively, while the corresponding Hausdorff95 distance of the proposed method reached 6.223, 7.033, and 4.411, respectively. The results of the comparison experiments demonstrated the obvious superior performance of the proposed method in segmenting ET and TC regions, especially that the performance of ET segmentation was the best in BraTS2020. Owing to the proposed two-stage segmentation scheme, with each having a feature selection module followed by a segmentation network, the potentially cross-modality-fused features could be automatically extracted from each modality of MR images, thus the performance of segmenting the three tumor regions was significantly improved.
[1] 葛婷, 牟宁, 李黎. 基于softmax回归与图割法的脑肿瘤分割算法[J]. 电子学报, 2017, 45(3): 644-649. [2] 杨一风, 胡颖, 聂生东. 基于磁共振图像的帕金森病计算机辅助诊断研究进展[J]. 中国生物医学工程学报, 2020, 39(5): 603-610. [3] 李锵, 白柯鑫, 赵柳, 等. MRI脑肿瘤图像分割研究进展及挑战[J]. 中国图象图形学报, 2020, 25(3): 419-431. [4] Gordillo N, Montseny E, Sobrevilla P. State of the art survey on MRI brain tumor segmentation [J]. Magnetic Resonance Imaging, 2013, 31(8): 1426-1438. [5] Wong KP. Medical image segmentation: methods and applications in functional imaging [M]// Handbook of Biomedical Image Analysis. Boston: Springer, 2015:111-182. [6] Farag A, Ahmed M, El-Baz A, et al. Advanced segmentation techniques [M]// Handbook of Biomedical Image Analysis. Boston: Springer, 2005: 479-533. [7] Pham DL, Xu Chenyang, Prince JL. Current methods in medical image segmentation [J]. Annual Review of Biomedical Engineering, 2000, 2(1):315-337. [8] Bauer S, Nolte LP, Reyes M. Segmentation of brain tumor images based on atlas-registration combined with a Markov-Random-Field lesion growth model [C]// IEEE International Symposium on Biomedical Imaging: from Nano to Macro. Chicago: IEEE, 2011: 2018-2021. [9] Wu Wei, Chen AYC, Zhao Liang. Brain tumor detection and segmentation in a CRF (conditional random fields) framework with pixel-pairwise affinity and superpixel-level features [J]. International Journal of Computer Assisted Radiology and Surgery, 2014, 9(2): 241-253. [10] Pinto A, Pereira S, Dinis H, et al. Random decision forests for automatic brain tumor segmentation on multi-modal MRI images [C]// 2015 IEEE 4th Portuguese Meeting on Bioengineering (ENBENG). Porto: IEEE, 2015: 1-5. [11] Havaei M, Dutil F, Pal C, et al. A convolutional neural network approach to brain tumor segmentation [C]// Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, BrainLes 2015. Cham: Springer, 2015: 195-208. [12] Pereira S, Pinto A, Alves V, et al. Brain tumor segmentation using convolutional neural networks in MRI images [J]. IEEE Transactions on Medical Imaging, 2016, 35(5):1240-1251. [13] Kamnitsas K, Ledig C, Newcombe V, et al. Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation [J]. Medical Image Analysis, 2016, 36: 61-78. [14] Zhao Liya, Jia Kebin. Multiscale CNNs for brain tumor segmentation and diagnosis [J]. Computational and Mathematical Methods in Medicine, 16 Mar, 2016 [Epub ahead of print]. [15] Chen Hao, Dou Qi, Yu Lequan, et al. VoxResNet: deep voxelwise residual networks for brain segmentation from 3D MR images [J]. NeuroImage, 2018, 170:446-455. [16] Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation [C]// Proceedings of 2015 IEEE Computer Vision and Pattern Recognition. Boston: IEEE, 2015: 3431-3440. [17] Ronneberger O, Fischer P, Brox T. U-Net: convolutional networks for biomedical image segmentation [C]// Medical Image Computing and Computer-Assisted Intervention. Cham: Springer, 2015:234-241. [18] Çiçek Ö, Abdulkadir A, Lienkamp SS, et al. 3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation [C]// Medical Image Computing and Computer-Assisted Intervention-MICCAI 2016. Cham: Springer, 2016: 424-432. [19] Milletari F, Navab N, Ahmadi SA. V-Net: Fully convolutional neural networks for volumetric medical image segmentation [C]// 2016 Fourth International Conference on 3D Vision. Stanford: IEEE, 2016: 565-571. [20] Shen Haocheng, Wang Ruixuan, Zhang Jianguo, et al. Multi-task fully convolutional network for brain tumour segmentation [C]// Proceedings of the 21st Annual Conference on Medical Image Understanding and Analysis. Edinburgh: Springer, 2017: 239-248. [21] Wang Guotai, Li Wenqi, Ourselin S, et al. Automatic brain tumor segmentation using cascaded anisotropic convolutional neural networks [C]// Proceedings of the 3rd International Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. Quebec City: Springer, 2018: 178-190. [22] Chen LC, Papandreou G, Kokkinos I, et al. Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFS [J]. IEEETrans on Pattern Analysis And Machine Intelligence, 2017, 40: 834-848. [23] He K, Zhang X, Ren S, et al. Identity mappings in deep residual networks [C]// European Conference on Computer Vision. Amsterdam: Springer, 2016: 630-645. [24] Stawiaski J. A multiscale patch based convolutional network for brain tumor segmentation [EB/OL]. https://arxiv.org/pdf/1710.0231601.pdf, 2019-10-03/2022-01-26. [25] Shaikh M, Anand G, Acharya G, et al. Brain tumor segmentation using dense fully convolutional neural network [C]// Proceedings of the 3rd International Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. Quebec City: Springer, 2018: 309-319. [26] Liu Di, Zhang Hao, Zhao Mingming, et al. Brain tumor segmention based on dilated convolution refine networks [C]// Proceedings of the 16th IEEE International Conference on Software Engineering Research, Management and Applications. Kunming: IEEE, 2018: 113-120. [27] 江宗康, 吕晓钢, 张建新, 等. MRI脑肿瘤图像分割的深度学习方法综述[J]. 中国图象图形学报, 2020, 25:215-228. [28] Beers A, Chang K, Brown J, et al. Sequential 3D U-Nets for biologically-informed brain tumor segmentation [EB/OL]. https://arxiv.org/pdf/1709.02967.pdf, 2019-07-27/2022-01-26. [29] Isensee F, Kickingereder P, Wick W, et al. Brain tumor segmentation and radiomics survival prediction: Contribution to the brats 2017 challenge [C]// Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, BrainLes 2017. Cham: Springer, 2018: 287-297. [30] Sherman R. A volumetric convolutional neural network for brain tumor segmentation [EB/OL]. https://arxiv.org/pdf/1811.0265401.pdf, 2019-10-03/2022-01-26. [31] Myronenko A. 3D MRI brain tumor segmentation using autoencoder regularization [C]// Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, BrainLes 2018. Granada: Springer, 2019: 311-320. [32] Li Xiang, Wang Wenhai, Hu Xiaolin, et al. Selective kernel network [C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2020: 510-519. [33] Menze B H, Jakab A, Bauer S, et al. The multimodal brain tumor image segmentation benchmark (BRATS) [J]. IEEE Transactions on Medical Imaging, 2014, 34(10): 1993-2024. [34] Bakas S, Akbari H, Sotiras A, et al. Advancing the cancer genome atlas glioma MRI collections with expert segmentation labels and radiomic features [J]. Nature Scientific Data, 2017, 4:170117. [35] Bakas S, Reyes M, Jakab A, et al. Identifying the best machine learning algorithms for brain tumor segmentation, progression assessment, and overall survival prediction in the BRATS challenge [C]// Radiomics and Radiogenomics: Technical Basis and Clinical Application. New York: Chapman and Hall/CRC, 2019: 99-114. [36] Isensee F, Kickingereder P, Wick W, et al. No New-Net [C]// Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, BrainLes 2018. Granada: Springer, 2019: 234-244. [37] McKinley R, Meier R, Wiest R. Ensembles of densely-connected CNNs with label-uncertainty for brain tumor segmentation [C]// Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, BrainLes 2018. Granada: Springer, 2019: 456-465. [38] Zhou Chenhong, Chen Shengcong, Ding Changxing, et al. Learning contextual and attentive information for brain tumor segmentation [C]// Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, BrainLes 2018. Granada: Springer, 2019: 497-507. [39] Kermi A, Mahmoudi I, Khadir MT. Deep convolutional neural networks using U-Net for automatic brain tumor segmentation in multimodal MRI volumes. [C]// Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, BrainLes 2018. Granada: Springer, 2019:37-48. [40] Feng Xue, Tustison N, Meyer C. Brain tumor segmentation using an ensemble of 3D U-Nets and overall survival prediction using radiomic features [C]// Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, BrainLes 2018. Granada: Springer, 2019: 279-288. [41] Puch S, Sánchez I, Hernández A, et al. Global planar convolutions for improved context aggregation in brain tumor segmentation [C]// Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, BrainLes 2018. Granada: Springer, 2019: 393-405. [42] Jiang Zeyu, Ding Changxing, Liu Minfeng, et al. Two-stage cascaded U-Net: 1st place solution to brats challenge 2019 segmentation task [C]// Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, BrainLes 2019. Cham: Springer, 2020: 231-241. [43] Zhao Yuanxing, Zhang Yanming, Liu Chenglin. Bag of tricks for 3D MRI brain tumor segmentation [C]// Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, BrainLes 2019. Cham: Springer, 2020: 210-220. [44] McKinley R, Rebsamen M, Meier R, et al. Triplanar ensemble of 3D-to-2D CNNS with label-uncertainty for brain tumor segmentation [C]// Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, BrainLes 2019. Cham: Springer, 2020:379-387. [45] Li Xiangyu, Luo Gongning, Wang Kuanquan. Multi-step cascaded networks for brain tumor segmentation [C]// Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, BrainLes 2019. Cham: Springer, 2020: 163-173. [46] Kim S, Luna M, Chikontwe P, et al. Two-step U-Nets for brain tumor segmentation and random forest with radiomics for survival time prediction [C]// Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, BrainLes 2019. Cham: Springer, 2020: 200-209. [47] Amian M, Soltaninejad M. Multi-resolution 3D CNN for MRI brain tumor segmentation and survival prediction [C]//Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, BrainLes 2019. Cham: Springer, 2020: 221-230. [48] Isensee F, Jäger PF, Full PM, et al. nnU-Net for brain tumor segmentation [C]// Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, BrainLes 2020. Cham: Springer, 2021: 118-132. [49] Jia Haozhe, Cai Weidong, Huang Heng, et al. H2NF-Net for brain tumor segmentation using multimodal MR imaging: 2nd place solution to brats challenge 2020 segmentation task [C]// Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, BrainLes 2020. Cham: Springer, 2021: 58-68. [50] Wang Yixin, Zhang Yao, Hou Feng, et al. Modality-Pairing learning for brain tumor segmentation [C]// Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, BrainLes 2020. Cham: Springer, 2021: 230-240. [51] Yuan Yading. Automatic brain tumor segmentation with scale attention network [C]//Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, BrainLes 2020. Cham: Springer, 2021:285-294. [52] Lyu Chenggang, Shu Hai. A two-Stage cascade model with variational autoencoders and attention gates for MRI brain tumor segmentation [C]// Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, BrainLes 2020. Cham: Springer, 2021: 435-447. [53] Pei Linmin, Murat AK, Colen R. Multimodal brain tumor segmentation and survival prediction using a 3D self-ensemble ResUNet [C]// Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, BrainLes 2020. Cham: Springer, 2021: 367-375. [54] Ghaffari M, Sowmya A, Oliver R. Automated brain tumour segmentation using cascaded 3D densely-connected U-Net [C]// Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, BrainLes 2020. Cham: Springer, 2021: 481-491.
