|
|
|
| Mask R-CNN and Data Augmentation and Transfer Learning |
| Wang Congzhi1&, Xu Zibi2&, Ma Xiangyuan1, Hong Zilan3, Fang Qiang1*, Guo Yanchun2* |
1(School of Biomedical Engineering, Shantou University, Shantou 515063, Guangdong, China)
2(Department of Neurosurgery, Second Affiliated Hospital of Medical College of Shantou University, Shantou 515063, Guangdong, China)
3(School of Biomedical Engineering, Shanghai Jiaotong University, Shanghai 200240, China) |
|
|
|
|
Abstract In clinical practices, the segmentation and modeling of brain regions in brain CT images can better observe the relationship between the lesion and the location of each organ. At present, the segmentation is mainly divided by manual outline, which is time-consuming, laborious and susceptible to subjective influence. In this paper, a Mask R-CNN based on augmentation and transfer learning was proposed, aiming to segment several brain regions vulnerable to cerebral hemorrhage from brain CT images more quickly and automatically, the regions included cerebellum, brainstem, basal ganglia region and dorsal thalamus. In this paper, 1 549 brain CT images of 100 cases of healthy people from July 2020 to December 2020 were analyzed. A total of 1 239 brain CT images of 80 cases were selected as the training set, and 310 brain CT images of the remaining 20 cases were selected as the test set. Then, the Mask R-CNN framework was used for training and prediction. Finally, the coordinates, names and masks of each brain region were output. To study the effect of data augmentation and transfer learning on model training, experiments of data augmentation and transfer learning were designed respectively, and the control group of U-NET model was designed. The data augmentation group expanded the training set to 13 629 images by means of rotation. In the transfer learning group, transfer learning was carried out based on the weights trained in MS-COCO. Among them, the transfer learning group had the best effect. In the experiment of transfer learning, the test set mAP was 0.909 7, the average IOU was 0.736 2, and the average DICE values of the test set of brain stem, cerebellum, basal ganglia region and dorsal thalamus were 0.902 5, 0.879 5, 0.781 8 and 0.828 4, respectively. The mAP and average IOU without data augmentation and transfer learning were 0.870 8 and 0.715 9, respectively. Data augmentation group were 0.894 1, 0.729 7; U-NET group were 0.839 0 and 0.671 1. These results showed that the Mask R-CNN convolutional neural network model could be used in the automatic segmentation of the common parts of cerebral hemorrhage, and the transfer learning greatly improved the training effect of the model.
|
|
Received: 27 February 2021
|
|
|
|
|
|
[1] Joanna MW, Colin S, Martin D. Mechanisms of sporadic cerebral small vessel disease: Insights from neuroimaging [J]. Lancet Neurology, 2013, 12(5): 483-497.
[2] 蒋小群,郝子龙,王秋筱,等. 脑出血患者出血部位与病因构成的相关性研究[J]. 中国脑血管病杂志, 2013, 10: 259-263.
[3] 纪海霞. 脑出血患者的发病原因与其脑出血部位的相关性[J]. 当代医药论丛, 2017, 15(19): 46-47.
[4] Torre V, Poggio T A. On edge detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1986, 8(2):147-163.
[5] 何俊,葛红,王玉峰.图像分割算法研究综述[J].计算机工程与科学,2009,31(12):58-61.
[6] 刘宇,陈胜. 医学图像分割方法综述[J]. 电子科技, 2017, 30(8): 169-172.
[7] Adams R, Bischof L. Seeded region growing [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1994, 16(6): 641-647.
[8] Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation[C]//Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention. Berlin: Springer, 2015: 234-241.
[9] 郑杨,梁光明,刘任任. 基于Mask R-CNN的宫颈细胞图像分割[J].计算机时代, 2020(10): 68-72.
[10] 张天麒,康波,孟祥飞,等. 基于U-net的颅内出血识别算法[J]. 北京邮电大学学报, 2020, 43(3): 92-98.
[11] 陈铭林. 基于深度学习的颅内出血CT影像分析[D]. 北京:中国科学院大学, 2020.
[12] Zhang Tianqi, Song Zheng, Yang Jianquan, et al. Cerebral hemorrhage recognition based on Mask R-CNN network [J]. Sensing and Imaging, 2021, 22(1): 1-16.
[13] Shakeri M, Tsogkas S, Ferrante E, et al. Sub-cortical brain structure segmentation using F-CNN′s[C]//2016 IEEE 13th International Symposium on Biomedical Imaging. Prague: IEEE, 2016: 269-272.
[14] 贺宝春,贾富仓. 基于组合U-net网络的CT图像头颈放疗危及器官自动分割[J]. 集成技术, 2020, 9(2): 17-24.
[15] 戴相昆,王小深,杜乐辉,等. 基于三维U-net深度卷积神经网络的头颈部危及器官的自动勾画[J]. 生物医学工程学杂志, 2020, 37(1): 136-141.
[16] 杨延武. 基于全卷积神经网络的脑结构分割方法探究[D]. 哈尔滨:哈尔滨工业大学, 2019.
[17] 李贞国. 脑部磁共振图像基底节区的局部分割研究[D]. 济南:山东大学, 2014.
[18] Iqbal A, Khan R, Karayannis T. Developing a brain atlas through deep learning[J]. Nature Machine Intelligence, 2019, 1: 277-287.
[19] He Kaiming, Zhang Xiangyu, Ren Shaoqing, et al. Deep residual learning for image recognition [C] ∥Proceedings of International Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778.
[20] He Kaiming, Gkioxari G, Dollar P, et al. Mask R-CNN[C] //Proceedings of the IEEE International Conference on Computer Vision. Venice: 2017: 2961-2969.
[21] Ren Shaoqing, He Kaming, R Girshick, et al. Faster R-CNN: Towards real-time object detection with region proposal net-works [J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017, 39(6): 1137-1149.
[22] Lin Tsung-Yi, Dollar P, Girshick R, et al. Feature pyramid networks for object detection[C] //2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE. 2017: 2117-2125.
[23] 林相泽,朱赛华,张俊媛,等. 基于迁移学习和Mask R-CNN的稻飞虱图像分类方法[J]. 农业机械学报, 2019, 50: 201-207.
[24] 朱有产,王雯瑶, 基于改进Mask R-CNN的绝缘子目标识别方法[J].微电子学与计算机, 2020, 37(2): 69-74.
[25] 林凯瀚,赵慧民,吕巨建,等. 基于Mask R-CNN的人脸检测与分割方法[J]. 计算机工程, 2020, 46(6): 274-280.
[26] 段仲静,李少波,胡建军,等. 基于Mask R-CNN的胶囊缺陷检测方法[J]. 无线电工程, 2020, 50(10): 857-862.
[27] Ben DS, Blitzer J, Crammer K, et al. Analysis of representations for domain adaptation[J]. Advances in Neural Information Processing Systems, 2007, 19: 137.
[28] Blitzer J, Mcdonald RT, Pereira F. Domain adaptation with structural correspondence learning[C] /Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing. Sydney: 2006: 120-128.
[29] Xing Dican, Dai Wenyuan, Xue Guirong, et al. Bridged refinement for transfer learning[C] //European Conference on Principles of Data Mining and Knowledge Discovery. Berlin: Springer, 2007: 324-335.
[30] 庄福振,罗平,何清,等. 迁移学习研究进展[J]. 软件学报, 2015, 26(1): 26-39.
[31] Lin Tsung-Yi, Maire M, Belongie S, et al. Microsoft COCO: Common objects in context[C] //European Conference on Computer Vision. Zurich: Springer International Publishing, 2014: 740-755.
[32] 侯小春. 基于卷积神经网络的目标检测算法研究[D]. 成都:电子科技大学, 2019.
[33] 王齐辉. 基于数据扩增的利用MRI先验图像的低剂量CT重建算法[D]. 上海:上海交通大学, 2019.
[34] Yao Yuanzhou, Zhao Yihang, Feng Ao, et al. Study on optimized lane detection algorithm based on U-net[C] //Proceedings of the 3rd International Conference on Computer Engineering, Information Science & Application Technology (ICCIA 2019). Nanchang: Atlantis Press, 2019: 237-244.
[35] 谢飞,穆昱,管子玉,等. 基于具有空间注意力机制的Mask R-CNN的口腔白斑分割[J]. 西北大学学报(自然科学版), 2020, 50(1): 9-15.
[36] 冯冬青. 基于深度学习的船只光学遥感图像检测和分割[D]. 成都:电子科技大学, 2019.
[37] 靳黎明. 开源项目推荐[J]. 程序员, 2007(10): 21-21.
[38] Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation[C] //International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich: Springer, Cham, 2015: 234-241.
[39] Rosenstein MT, Marx Z, Kaelbling LP, et al. To transfer or not to transfer[C] //NIPS 2005 Workshop on Transfer Learning. Vancouver: 2005, 898: 1-4.
[40] Dai Wenyuan, Jin Ou, Xue Guirong, et al. EigenTransfer: A unified framework for transfer learning[C]//International Conference on Machine Learning. Montreal: ACM, 2009: 193-200.
[41] 李永盛,何佳洲,赵国清,等. 关于迁移学习中的负迁移方向研究[J]. 指挥控制与仿真, 2020, 42(4): 28-33.
[42] 李宗桂,张俊华,梅礼晔. 基于Mask R-CNN的超声图像中胎儿头围测量方法[J]. 中国生物医学工程学报, 2021, 40: 12-18.
[43] 李思穆. 基于Mask R-CNN的前列腺TRUS图像分割方法研究[D].天津:天津工业大学, 2020.
[44] 储春洁,王佳雯,韩雅琪,等. 基于Mask R-CNN模型的胸片肺结节检测性能评估[J]. 信息与控制, 2020, 49(6): 728-734.
[45] Hasegawa R, Iwamoto Y, Lin L, et al. Automatic segmentation of liver tumor in multiphase CT images by Mask R-CNN[C]// 2020 IEEE 2nd Global Conference on Life Sciences and Technologies (LifeTech). Kyoto: IEEE, 2020: 231-234.
[46] Hu Q, Souza LFF, Holanda GB, et al. An effective approach for CT lung segmentation using mask region-based convolutional neural networks [J]. Artificial Intelligence in Medicine, 2020, 103: 101792. |
|
|
|
