基于数据扩增和迁移学习的Mask R-CNN脑CT图像自动分割研究

doi:10.3969/j.issn.0258-8021.2021.04.04

摘要
图/表
参考文献
相关文章 (7)

全文: PDF (3896 KB) HTML (1 KB)
输出: BibTeX | EndNote (RIS)

摘要在临床上,分割脑CT图像上的各个脑区并建模,可以更好地观察病变与各器官的位置关系。目前医生主要通过人工勾画的方式进行分割,不仅费时、费力,而且还容易受主观因素影响。提出一种基于数据扩增和迁移学习的Mask R-CNN卷积神经网络算法,可以更快速地在脑CT图像上自动分割出易受脑出血危及的脑区,如小脑、脑干、基底节区和背侧丘脑等部位。针对2020年7月—2020年12月的100名健康人的1 549张脑CT图像,选取其中的80例,共1 239张图像作为训练集,其余的20例,共310张图像作为测试集,然后采用Mask R-CNN框架进行训练和预测,最终输出各脑区的坐标、名称与mask掩模。为研究数据扩增和迁移学习对模型训练效果的提升,分别设计了数据扩增和迁移学习的实验,以及U-Net模型的对照组。数据扩增组通过旋转的方式进行数据扩增,将训练集扩增至13 629张图像;迁移学习组在MS-COCO上训练好的权重基础上,进行迁移学习。其中,迁移学习组的效果最好。在迁移学习的实验中,测试集mAP为0.909 7,平均IOU为0.736 2,脑干、小脑、基底节区和背侧丘脑的测试集平均DICE值分别为0.902 5、0.879 5、0.781 8、0.828 4。而未进行数据扩增和迁移学习的mAP和平均IOU分别为0.870 8、0.715 9;数据扩增组则为0.894 1、0.729 7;U-Net组则为0.839 0、0.671 1。研究显示,Mask R-CNN卷积神经网络模型可以较好地应用于脑出血常见部位的自动分割,且迁移学习对模型训练效果的提升较大。

	服务

	把本文推荐给朋友
	加入我的书架
	加入引用管理器
	E-mail Alert
	RSS
	作者相关文章
	王琮智
	许梓璧
	马祥园
	洪子澜
	方强
	郭燕春

关键词 ：数据扩增, 迁移学习, Mask R-CNN, 自动分割

Abstract：In clinical practices, the segmentation and modeling of brain regions in brain CT images can better observe the relationship between the lesion and the location of each organ. At present, the segmentation is mainly divided by manual outline, which is time-consuming, laborious and susceptible to subjective influence. In this paper, a Mask R-CNN based on augmentation and transfer learning was proposed, aiming to segment several brain regions vulnerable to cerebral hemorrhage from brain CT images more quickly and automatically, the regions included cerebellum, brainstem, basal ganglia region and dorsal thalamus. In this paper, 1 549 brain CT images of 100 cases of healthy people from July 2020 to December 2020 were analyzed. A total of 1 239 brain CT images of 80 cases were selected as the training set, and 310 brain CT images of the remaining 20 cases were selected as the test set. Then, the Mask R-CNN framework was used for training and prediction. Finally, the coordinates, names and masks of each brain region were output. To study the effect of data augmentation and transfer learning on model training, experiments of data augmentation and transfer learning were designed respectively, and the control group of U-NET model was designed. The data augmentation group expanded the training set to 13 629 images by means of rotation. In the transfer learning group, transfer learning was carried out based on the weights trained in MS-COCO. Among them, the transfer learning group had the best effect. In the experiment of transfer learning, the test set mAP was 0.909 7, the average IOU was 0.736 2, and the average DICE values of the test set of brain stem, cerebellum, basal ganglia region and dorsal thalamus were 0.902 5, 0.879 5, 0.781 8 and 0.828 4, respectively. The mAP and average IOU without data augmentation and transfer learning were 0.870 8 and 0.715 9, respectively. Data augmentation group were 0.894 1, 0.729 7; U-NET group were 0.839 0 and 0.671 1. These results showed that the Mask R-CNN convolutional neural network model could be used in the automatic segmentation of the common parts of cerebral hemorrhage, and the transfer learning greatly improved the training effect of the model.

Key words： data augmentation transfer learning Mask R-CNN automatic segmentation

收稿日期: 2021-02-27

PACS:

R318

基金资助:2020李嘉诚基金会交叉科研项目(2020LKSFG01C).

通讯作者: * E-mail: qiangfang@stu.edu.cn;109555380@qq.com

作者简介: ^&共同第一作者(Co-first author)

引用本文:

王琮智, 许梓璧, 马祥园, 洪子澜, 方强, 郭燕春. 基于数据扩增和迁移学习的Mask R-CNN脑CT图像自动分割研究[J]. 中国生物医学工程学报, 2021, 40(4): 410-418.
Wang Congzhi, Xu Zibi, Ma Xiangyuan, Hong Zilan, Fang Qiang, Guo Yanchun. Mask R-CNN and Data Augmentation and Transfer Learning. Chinese Journal of Biomedical Engineering, 2021, 40(4): 410-418.

链接本文:

http://cjbme.csbme.org/CN/10.3969/j.issn.0258-8021.2021.04.04 或 http://cjbme.csbme.org/CN/Y2021/V40/I4/410

[1] Joanna MW, Colin S, Martin D. Mechanisms of sporadic cerebral small vessel disease: Insights from neuroimaging [J]. Lancet Neurology, 2013, 12(5): 483-497.
[2] 蒋小群,郝子龙,王秋筱,等. 脑出血患者出血部位与病因构成的相关性研究[J]. 中国脑血管病杂志, 2013, 10: 259-263.
[3] 纪海霞. 脑出血患者的发病原因与其脑出血部位的相关性[J]. 当代医药论丛, 2017, 15(19): 46-47.
[4] Torre V, Poggio T A. On edge detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1986, 8(2):147-163.
[5] 何俊,葛红,王玉峰.图像分割算法研究综述[J].计算机工程与科学,2009,31(12):58-61.
[6] 刘宇,陈胜. 医学图像分割方法综述[J]. 电子科技, 2017, 30(8): 169-172.
[7] Adams R, Bischof L. Seeded region growing [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1994, 16(6): 641-647.
[8] Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation[C]//Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention. Berlin: Springer, 2015: 234-241.
[9] 郑杨,梁光明,刘任任. 基于Mask R-CNN的宫颈细胞图像分割[J].计算机时代, 2020(10): 68-72.
[10] 张天麒,康波,孟祥飞,等. 基于U-net的颅内出血识别算法[J]. 北京邮电大学学报, 2020, 43(3): 92-98.
[11] 陈铭林. 基于深度学习的颅内出血CT影像分析[D]. 北京:中国科学院大学, 2020.
[12] Zhang Tianqi, Song Zheng, Yang Jianquan, et al. Cerebral hemorrhage recognition based on Mask R-CNN network [J]. Sensing and Imaging, 2021, 22(1): 1-16.
[13] Shakeri M, Tsogkas S, Ferrante E, et al. Sub-cortical brain structure segmentation using F-CNN′s[C]//2016 IEEE 13th International Symposium on Biomedical Imaging. Prague: IEEE, 2016: 269-272.
[14] 贺宝春,贾富仓. 基于组合U-net网络的CT图像头颈放疗危及器官自动分割[J]. 集成技术, 2020, 9(2): 17-24.
[15] 戴相昆,王小深,杜乐辉,等. 基于三维U-net深度卷积神经网络的头颈部危及器官的自动勾画[J]. 生物医学工程学杂志, 2020, 37(1): 136-141.
[16] 杨延武. 基于全卷积神经网络的脑结构分割方法探究[D]. 哈尔滨:哈尔滨工业大学, 2019.
[17] 李贞国. 脑部磁共振图像基底节区的局部分割研究[D]. 济南:山东大学, 2014.
[18] Iqbal A, Khan R, Karayannis T. Developing a brain atlas through deep learning[J]. Nature Machine Intelligence, 2019, 1: 277-287.
[19] He Kaiming, Zhang Xiangyu, Ren Shaoqing, et al. Deep residual learning for image recognition [C] ∥Proceedings of International Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778.
[20] He Kaiming, Gkioxari G, Dollar P, et al. Mask R-CNN[C] //Proceedings of the IEEE International Conference on Computer Vision. Venice: 2017: 2961-2969.
[21] Ren Shaoqing, He Kaming, R Girshick, et al. Faster R-CNN: Towards real-time object detection with region proposal net-works [J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017, 39(6): 1137-1149.
[22] Lin Tsung-Yi, Dollar P, Girshick R, et al. Feature pyramid networks for object detection[C] //2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE. 2017: 2117-2125.
[23] 林相泽,朱赛华,张俊媛,等. 基于迁移学习和Mask R-CNN的稻飞虱图像分类方法[J]. 农业机械学报, 2019, 50: 201-207.
[24] 朱有产,王雯瑶, 基于改进Mask R-CNN的绝缘子目标识别方法[J].微电子学与计算机, 2020, 37(2): 69-74.
[25] 林凯瀚,赵慧民,吕巨建,等. 基于Mask R-CNN的人脸检测与分割方法[J]. 计算机工程, 2020, 46(6): 274-280.
[26] 段仲静,李少波,胡建军,等. 基于Mask R-CNN的胶囊缺陷检测方法[J]. 无线电工程, 2020, 50(10): 857-862.
[27] Ben DS, Blitzer J, Crammer K, et al. Analysis of representations for domain adaptation[J]. Advances in Neural Information Processing Systems, 2007, 19: 137.
[28] Blitzer J, Mcdonald RT, Pereira F. Domain adaptation with structural correspondence learning[C] /Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing. Sydney: 2006: 120-128.
[29] Xing Dican, Dai Wenyuan, Xue Guirong, et al. Bridged refinement for transfer learning[C] //European Conference on Principles of Data Mining and Knowledge Discovery. Berlin: Springer, 2007: 324-335.
[30] 庄福振,罗平,何清,等. 迁移学习研究进展[J]. 软件学报, 2015, 26(1): 26-39.
[31] Lin Tsung-Yi, Maire M, Belongie S, et al. Microsoft COCO: Common objects in context[C] //European Conference on Computer Vision. Zurich: Springer International Publishing, 2014: 740-755.
[32] 侯小春. 基于卷积神经网络的目标检测算法研究[D]. 成都:电子科技大学, 2019.
[33] 王齐辉. 基于数据扩增的利用MRI先验图像的低剂量CT重建算法[D]. 上海:上海交通大学, 2019.
[34] Yao Yuanzhou, Zhao Yihang, Feng Ao, et al. Study on optimized lane detection algorithm based on U-net[C] //Proceedings of the 3rd International Conference on Computer Engineering, Information Science & Application Technology (ICCIA 2019). Nanchang: Atlantis Press, 2019: 237-244.
[35] 谢飞,穆昱,管子玉,等. 基于具有空间注意力机制的Mask R-CNN的口腔白斑分割[J]. 西北大学学报(自然科学版), 2020, 50(1): 9-15.
[36] 冯冬青. 基于深度学习的船只光学遥感图像检测和分割[D]. 成都:电子科技大学, 2019.
[37] 靳黎明. 开源项目推荐[J]. 程序员, 2007(10): 21-21.
[38] Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation[C] //International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich: Springer, Cham, 2015: 234-241.
[39] Rosenstein MT, Marx Z, Kaelbling LP, et al. To transfer or not to transfer[C] //NIPS 2005 Workshop on Transfer Learning. Vancouver: 2005, 898: 1-4.
[40] Dai Wenyuan, Jin Ou, Xue Guirong, et al. EigenTransfer: A unified framework for transfer learning[C]//International Conference on Machine Learning. Montreal: ACM, 2009: 193-200.
[41] 李永盛,何佳洲,赵国清,等. 关于迁移学习中的负迁移方向研究[J]. 指挥控制与仿真, 2020, 42(4): 28-33.
[42] 李宗桂,张俊华,梅礼晔. 基于Mask R-CNN的超声图像中胎儿头围测量方法[J]. 中国生物医学工程学报, 2021, 40: 12-18.
[43] 李思穆. 基于Mask R-CNN的前列腺TRUS图像分割方法研究[D].天津:天津工业大学, 2020.
[44] 储春洁,王佳雯,韩雅琪,等. 基于Mask R-CNN模型的胸片肺结节检测性能评估[J]. 信息与控制, 2020, 49(6): 728-734.
[45] Hasegawa R, Iwamoto Y, Lin L, et al. Automatic segmentation of liver tumor in multiphase CT images by Mask R-CNN[C]// 2020 IEEE 2nd Global Conference on Life Sciences and Technologies (LifeTech). Kyoto: IEEE, 2020: 231-234.
[46] Hu Q, Souza LFF, Holanda GB, et al. An effective approach for CT lung segmentation using mask region-based convolutional neural networks [J]. Artificial Intelligence in Medicine, 2020, 103: 101792.