基于Transformer深度学习模型在医学图像分割中的研究进展

doi:10.3969/j.issn.0258-8021.2024.04.009

摘要
图/表
参考文献
相关文章 (15)

全文: PDF (1000 KB) HTML (1 KB)
输出: BibTeX | EndNote (RIS)

摘要医学图像的准确分割在现代临床影像检查、精准诊断和治疗规划中意义至关重要。近10年来,卷积神经网络(CNN)凭借其独特的特征提取能力,在医学图像分割领域成绩显著。CNN架构中存在的局部感受野和固有归纳偏置局限,限制其对图像中远程依赖关系的有效建模。近年来,Transformer架构依赖其对全局信息的捕获能力,有助于建模长距离的依赖关系并挖掘语义信息,在生物医学图像分割领域展示出卓越的性能和巨大潜力。在此,对Transformer架构的组成及其在医学图像分割中的应用进行了全面综述,从全监督、无监督和半监督的角度出发,对Transformer架构在医学图像的腹部多器官分割、心脏分割和脑肿瘤分割中的运用价值及性能进行了归纳分析,并对Transformer模型在分割任务中存在的局限不足进行了概括总结,最后对其未来发展趋势及优化路径进行了探讨展望。

	服务

	把本文推荐给朋友
	加入我的书架
	加入引用管理器
	E-mail Alert
	RSS
	作者相关文章
	周腊珍
	陈红池
	李秋霞
	李坊佐

关键词 ： Transformer, 图像分割, 卷积神经网络, 医学图像

Abstract：Accurate segmentation of medical images is a crucial step in clinical diagnosis and treatment. Over the past decade, convolutional neural network (CNN) has been widely applied in the field of medical image segmentation and have achieved excellent segmentation performance. However, the inherent inductive bias in CNN architectures limits their ability to model long-range dependencies in images. In contrast, the Transformer architectures, which focus on global information and the ability to model long-range dependencies, has been demonstrated outstanding performance in biomedical image segmentation. This review introduced the components of Transformer architecture and its applications in medical image segmentation. From perspectives of fully supervised, unsupervised and semi-supervised learning, application values and performances of Transformer architectures in abdominal multi-organ segmentation, cardiac segmentation and brain tumor segmentation were summarized and analyzed. Finally, limitations of Transformer model in segmentation tasks and future optimizations were prospected.

Key words： Transformer image segmentation convolutional neural network medical image

收稿日期: 2023-09-06

PACS:

R318

基金资助:国家自然科学基金(11865003);江西省自然科学基金(20224BAB201020);赣南医科大学科研启动基金(QD201805)

通讯作者: ^*E-mail: lfz880920@163.com

引用本文:

周腊珍, 陈红池, 李秋霞, 李坊佐. 基于Transformer深度学习模型在医学图像分割中的研究进展[J]. 中国生物医学工程学报, 2024, 43(4): 467-476.
Zhou Lazhen, Chen Hongchi, Li Qiuxia, Li Fangzuo. Research Progress on Transformer-Based Deep Learning Models for Medical Image Segmentation. Chinese Journal of Biomedical Engineering, 2024, 43(4): 467-476.

链接本文:

http://cjbme.csbme.org/CN/10.3969/j.issn.0258-8021.2024.04.009 或 http://cjbme.csbme.org/CN/Y2024/V43/I4/467

[1] Zhou Shaohua, Greenspan H, Davatzikos C, et al. A review of deep learning in medical imaging: imaging traits, technology trends, case studies with progress highlights, and future promises[J]. Proceedings of the IEEE, 2021, 109(5): 820-838.
[2] Beutel J, Kundel HL, Van Metter RL. Handbook of Medical Imaging[M]. Bellingham: SPEI Press, 2000.
[3] Lo SB, Lou SA, Lin JS, et al. Artificial convolution neural network techniques and applications for lung nodule detection[J]. IEEE Trans Med Imaging, 1995, 14(4): 711-718.
[4] Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation[C]//Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference. Munich: Springer International Publishing, 2015: 234-241.
[5] Seo H, Huang C, Bassenne M, et al. Modified U-Net (mU-Net) with incorporation of object-dependent high level features for improved liver and liver-tumor segmentation in CT images[J]. IEEE Trans Med Imaging, 2020, 39(5): 1316-1325.
[6] Cheng Yinlin, Ma Mengnan, Zhang Liangjun, et al. Retinal blood vessel segmentation based on densely connected U-Net[J]. Math Biosci Eng, 2020, 17(4): 3088-3108.
[7] 纪建兵, 陈纾, 杨媛媛. 双重降维通道注意力门控U-Net的胰腺CT分割[J]. 中国生物医学工程学报, 2023, 42(3): 281-288.
[8] Zhou Hongyu, Lu Chixiang, Yang Sibei, et al. Convnets vs transformers: whose visual representations are more transferable?[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Montreal: IEEE, 2021: 2230-2238.
[9] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition[DB/OL]. https://arxiv.org/abs/1409.1556, 2015-04-10/2023-09-01.
[10] Liang Ming, Hu Xiaolin. Recurrent convolutional neural network for object recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Boston: IEEE, 2015: 3367-3375.
[11] He Kaiming, Zhang Xiangyu, Ren Shaoqing, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778.
[12] Devalla SK, Renukanand PK, Sreedhar BK, et al. DRUNET: a dilated-residual u-net deep learning network to segment optic nerve head tissues in optical coherence tomography images[J]. Biomed Opt Express, 2018, 9(7): 3244-3265.
[13] Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[C]//Advances in Neural Information Processing Systems. New York: Curran Associates Inc, 2017: 5998-6008.
[14] Wang Wenxuan, Chen Chen, Ding Meng, et al. Transbts: multimodal brain tumor segmentation using transformer[C]//Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference. Strasbourg: Springer International Publishing, 2021: 109-119.
[15] Xu Guoping, Wu Xingrong, Zhang Xuan, et al. Levit-unet: make faster encoders with transformer for medical image segmentation[DB/OL]. https://arxiv.org/abs/2107.08623, 2021-07-19/2023-09-01.
[16] Hong Zhifang, Chen Mingzhi, Hu Weijie, et al. Dual encoder network with transformer-CNN for multi-organ segmentation[J]. Med Biol Eng Comput, 2023, 61(3): 661-671.
[17] Chen Jieneng, Lu Yongyi, Yu Qihang, et al. Transunet: transformers make strong encoders for medical image segmentation[DB/OL]. https://arxiv.org/abs/2102.04306, 2021-02-08/2023-09-01.
[18] 李军, 叶欣怡, 杨长才, 等. 基于联合深度网络和形态结构约束的三维医学图像分割方法[J]. 中国生物医学工程学报, 2023, 42(1): 30-40.
[19] Liu Ze, Lin Yutong, Cao Yue, et al. Swin transformer: Hierarchical vision transformer using shifted windows[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). Montreal: IEEE, 2021: 10012-10022.
[20] Cao Hu, Wang Yueyue, Chen J, et al. Swin-unet: Unet-like pure transformer for medical image segmentation[C]//European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022: 205-218.
[21] Wu Yixuan, Liao Kuanlun, Chen Jintai, et al. D-former: a U-shaped dilated transformer for 3D medical image segmentation[J]. Neural Computing and Applications, 2023, 35(2): 1931-1944.
[22] Zhou Hongyu, Guo Jiansen, Zhang Yinghao, et al. Nnformer: interleaved transformer for volumetric segmentation[DB/OL]. https://arxiv.org/abs/2109.03201, 2022-02-04/2023-09-01.
[23] Shaker A, Maaz M, Rasheed H, et al. UNETR++: delving into efficient and accurate 3D medical image segmentation[DB/OL]. https://arxiv.org/abs/2212.04497, 2023-03-22/2023-09-01.
[24] Azad R, Arimond R, Aghdam EK, et al. Dae-former: dual attention-guided efficient transformer for medical image segmentation[DB/OL]. https://arxiv.org/abs/2212.13504, 2023-07-26/2023-09-01.
[25] Yang XN, Tian XL. Transnunet: using attention mechanism for whole heart segmentation[C]//2022 IEEE 2nd International Conference on Power, Electronics and Computer Applications (ICPECA). Shenyang: IEEE, 2022: 553-556.
[26] Woo S, Park J, Lee JY, et al. Cbam: convolutional block attention module[DB/OL]. https://arxiv.org/abs/1807.06521, 2018-07-18/2023-09-01.
[27] Wang Haonan, Cao Peng, Wang Jiaqi, et al. UCTransnet: rethinking the skip connections in u-net from a channel-wise perspective with transformer[C]//Proceedings of the AAAI Conference on Artificial Intelligence. California: AAAI Press, 2022: 2441-2449.
[28] Huang Xiaohong, Deng Zhifang, Li Dandan, et al. MISSFormer: an effective transformer for 2D medical image segmentation[J]. IEEE Trans Med Imaging, 2023, 42(5): 1484-1494.
[29] Wang Jing, Zhao Haiyue, Liang Wei, et al. Cross-convolutional transformer for automated multi-organs segmentation in a variety of medical images[J]. Phys Med Biol, 2023, 68(3): 035008.
[30] Tragakis A, Kaul C, Murray-Smith R, et al. The fully convolutional transformer for medical image segmentation[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision(WACV). Waikoloa: IEEE, 2023: 3660-3669.
[31] Isensee F, Jaeger PF, Kohl SAA, et al. nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation[J]. Nature Methods, 2021, 18(2): 203-211.
[32] Hatamizadeh A, Tang Yucheng, Nath V, et al. UNETR: transformers for 3D medical image segmentation[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision(WACV). Waikoloa: IEEE, 2022: 574-584.
[33] Hatamizadeh A, Nath V, Tang Yucheng, et al. Swin unetr: swin transformers for semantic segmentation of brain tumors in mri images[C]//International MICCAI Brainlesion Workshop. Cham: Springer International Publishing, 2021: 272-284.
[34] Li Jiangyun, Wang Wenxuan, Chen Chen, et al. TransBTSV2: wider instead of deeper transformer for medical image segmentation[DB/OL]. https://arxiv.org/abs/2201.12785v2, 2022-05-17/2023-09-01.
[35] Peiris H, Hayat M, Chen Zhaolin, et al. A robust volumetric transformer for accurate 3D tumor segmentation[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention(MICCAI 2022). Cham: Springer Nature Switzerland, 2022: 162-172.
[36] Li Zongren, Wushouer S, Wang Yuzhen, et al. DenseTrans: multimodal brain tumor segmentation using swin transformer[J]. IEEE Access, 2023, 11: 42895-42908.
[37] Zhou Zongwei, Rahman Siddiquee MM, Tajbakhsh N, et al. Unet++: a nested u-net architecture for medical image segmentation[C]//Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support: 4th International Workshop, DLMIA 2018, and 8th International Workshop, ML-CDS 2018, Held in Conjunction with MICCAI 2018. Granada: Springer, 2018: 3-11.
[38] Liu Lihao, Huang Zhening, Liò P, et al. PC-SwinMorph: Patch representation for unsupervised medical image registration and segmentation[DB/OL]. https://arxiv.org/abs/2203.05684, 2022-07-20/2023-09-01.
[39] Wang Yiqing, Li Zihan, Mei Jieru, et al. Swinmm: masked multi-view with swin transformers for 3D medical image segmentation[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer Nature Switzerland, 2023: 486-496.
[40] Wang Tao, Lu Jianglin, Lai Zhihui, et al. Uncertainty-guided pixel contrastive learning for semi-supervised medical image segmentation[C]//Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence. Vienna: IJCAI, 2022: 1444-1450.
[41] Liu Qianying, Gu Xiao, Henderson P, et al. Multi-scale cross contrastive learning for semi-supervised medical image segmentation[DB/OL]. https://arxiv.org/abs/2306.14293, 2023-06-25/2023-09-01.
[42] Xie Yutong, Zhang Jianpeng, Xia Yong, et al. Unimiss: universal medical self-supervised learning via breaking dimensionality barrier[C]//European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022: 558-575.
[43] Wang Bo, Li Qian, You Zheng. Self-supervised learning based transformer and convolution hybrid network for one-shot organ segmentation[J]. Neurocomputing, 2023, 527: 1-12.
[44] Xiao Zhiyong, Su Yixin, Deng Zhaohong, et al. Efficient combination of CNN and transformer for dual-teacher uncertainty-guided semi-supervised medical image segmentation[J]. Computer Methods and Programs in Biomedicine, 2022, 226: 107099.
[45] Xie Shiao, Huang Huimin, Niu Ziwei, et al. MedFCT: a frequency domain joint CNN-transformer network for semi-supervised medical image segmentation[C]//2023 IEEE International Conference on Multimedia and Expo (ICME). Brisbane: IEEE, 2023: 1913-1918.
[46] Wu Wenxia, Yan Jing, Zhao Yuanshen, et al. Multi-task learning for concurrent survival prediction and semi-supervised segmentation of gliomas in brain MRI[J]. Displays, 2023, 78: 102402.
[47] Luo Xiangde, Hu Minhao, Song Tao, et al. Semi-supervised medical image segmentation via cross teaching between CNN and transformer[C]//International Conference on Medical Imaging with Deep Learning. Zurich: PMLR, 2022: 820-833.
[48] Shin H, Kim H, Kim S, et al. SDC-UDA: volumetric unsupervised domain adaptation framework for slice-direction continuous cross-modality medical image segmentation[DB/OL]. https://arxiv.org/abs/2305.11012, 2023-05-18/2023-09-01.
[49] Tang Yucheng, Yang Dong, Li Wenqi, et al. Self-supervised pre-training of swin transformers for 3d medical image analysis[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022: 20730-20740.
[50] Liang Junjie, Yang Cihui, Zhong Jingting, et al. Btswin-unet: 3D U-shaped symmetrical swin transformer-based network for brain tumor segmentation with self-supervised pre-training[J]. Neural Processing Letters, 2023, 55(4): 3695-3713.