Review on Applications of U-Net and its Variants in Medical Image Segmentation
Huang Xiaoming1, He Fuyun1,2,3*, Tang Xiaohu1, Wang Xun1,2, Qiu Senhui1,3, Hu Cong2
1(College of Electronic Engineering, Guangxi Normal University, Guilin 541004, China) 2(Guangxi Key Laboratory of Automatic Detection Technology and Instrument, Guilin 541004, China) 3(Guangxi Key Laboratory of Wireless Wideband Communication and Signal Processing, Guilin 541004, China)
Abstract：Medical image segmentation can provide a reliable basis for clinical diagnosis and pathology research and assist doctors to make accurate diagnosis. The emergence of medical image segmentation based on deep learning has solved the problems of low robustness and low accuracy in traditional automatic segmentation methods, among which, U-Net stands out among many segmentation networks with its excellent performance. Researchers have successively proposed a variety of improved variants based on U-Net. Taking U-Net and its network variants as the main content, this article first introduced the network structure and common improvement methods of U-Net in detail. Then, divided the U-Net variants into general-purpose networks and specific network according to the different segmentation objects, and discussed the research progress of these networks in the medical image segmentation. At the end, the difficulties and problems existing in this research field were analyzed, and the development directions were prospected.
黄晓鸣, 何富运, 唐晓虎, 王勋, 丘森辉, 胡聪. U-Net及其变体在医学图像分割中的应用研究综述[J]. 中国生物医学工程学报, 2022, 41(5): 567-576.
Huang Xiaoming,He Fuyun,*,Tang Xiaohu,Wang Xun,Qiu Senhui,Hu Cong. Review on Applications of U-Net and its Variants in Medical Image Segmentation. Chinese Journal of Biomedical Engineering, 2022, 41(5): 567-576.
[1] Ciresan D, Giusti A, Gambardella L, et al. Deep neural networks segment neuronal membranes in electron microscopy images [C]// Advances in Neural Information Processing Systems. Lake Tahoe: Curran Associates, 2012, 25: 2843-2851. [2] Zhao Xiaomei, Wu Yihong, Song Guidong, et al. A deep learning model integrating FCNNs and CRFs for brain tumor segmentation [J]. Medical Image Analysis, 2018, 43: 98-111. [3] Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation [C]//2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston: IEEE, 2015: 3431-3440. [4] Ronneberger O, Fischer P, Brox T. U-Net: Convolutional networks for biomedical image segmentation [C]//International Conference on Medical Image Computing and Computer-assisted Intervention. Munich: Springer, 2015: 234-241. [5] Zhou Zongwei, Siddiquee MMR, Tajbakhsh N, et al. UNet++: A nested U-Net architecture for medical image segmentation [C]//Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support. Granada: Springer, 2018: 3-11. [6] Alom MZ, Yakopcic C, Hasan M, et al. Recurrent residual U-Net for medical image segmentation [J]. Journal of Medical Imaging, 2019, 6(1): 014006. [7] Ibtehaz N, Rahman MS. MultiResUNet: rethinking the U-Net architecture for multimodal biomedical image segmentation [J]. Neural Networks, 2020, 121: 74-87. [8] Li Qiufu, Shen Linlin. 3D neuron reconstruction in tangled neuronal image with deep networks [J]. IEEE Transactions on Medical Imaging, 2020, 39(2): 425-435. [9] Nemoto T, Futakami N, Yagi M, et al. Efficacy evaluation of 2D, 3D U-Net semantic segmentation and atlas-based segmentation of normal lungs excluding the trachea and main bronchi [J]. Journal of Radiation Research, 2019, 61(1): 257-264. [10] Çiçek Ö, Abdulkadir A, Lienkamp SS, et al. 3D U-Net: Learning dense volumetric segmentation from sparse annotation [C]//International Conference on Medical Image Computing and Computer-assisted Intervention. Athens: Springer, 2016: 424-432. [11] Drozdzal M, Vorontsov E, Chartrand G, et al. The importance of skip connections in biomedical image segmentation [C]//Deep Learning and Data Labeling for Medical Applications. Athens: Springer, 2016: 179-187. [12] Wu Jiong, Zhang Yue, Wang Kai, et al. Skip connection U-Net for white matter hyperintensities segmentation from MRI [J]. IEEE Access, 2019, 7: 155194-155202. [13] Li Xiang, Jiang Yuchen, Li Minglei, et al. Lightweight attention convolutional neural network for retinal vessel image segmentation [J]. IEEE Transactions on Industrial Informatics, 2021, 17(3): 1958-1967. [14] He Kaiming, Zhang Xiangyu, Ren Shaoqing, et al. Deep residual learning for image recognition [C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778. [15] 郝华颖, 赵昆, 苏攀, 等. 一种基于改进ResU-Net的角膜神经分割算法[J]. 计算机工程, 2021, 47(1):217-223. [16] Huang Gao, Liu Zhuang, Van Der Maaten L, et al. Densely connected convolutional networks [C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 2261-2269. [17] Li Xiaomeng, Chen Hao, Qi Xiaojuan, et al. H-DenseNet: hybrid densely connected UNet for liver and tumor segmentation from CT volumes [J]. IEEE Transactions on Medical Imaging, 2018, 37(12): 2663-2674. [18] Chen Liang, Bentley P, Mori K, et al. DRINet for medical image segmentation [J]. IEEE Transactions on Medical Imaging, 2018, 37(11): 2453-2462. [19] Li Haoming, Fang Jinghui, Liu Shengfeng, et al. CR-Unet: a composite network for ovary and follicle segmentation in ultrasound images [J]. IEEE Journal of Biomedical and Health Informatics, 2020, 24(4): 974-983. [20] Liu Yiwei, Qi Na, Zhu Qing, et al. CR-U-Net: Cascaded U-Net with residual mapping for liver segmentation in CT Images [C]//2019 IEEE Visual Communications and Image Processing. Sydney: IEEE, 2019: 1-4. [21] Chen Jianxu, Yang Lin, Zhang Yizhe, et al. Combining fully convolutional and recurrent neural networks for 3D biomedical image segmentation [C]//Advances in Neural Information Processing Systems. Barcelona: Curran Associates, 2016, 29: 3044-3052. [22] Gu Zaiwang, Cheng Jun, Fu Huazhu, et al. CE-Net: Context encoder network for 2D medical image segmentation [J]. IEEE Transactions on Medical Imaging, 2019, 38(10): 2281-2292. [23] Huang Huimin, Lin Lanfen, Tong Ruofeng, et al. UNet 3+: a full-scale connected UNet for medical image segmentation [C]// 2020 IEEE International Conference on Acoustics, Speech and Signal Processing. Barcelona: IEEE, 2020: 1055-1059. [24] Kholiavchenko M, Sirazitdinov I, Kubrak K, et al. Contour-aware multi-label chest X-ray organ segmentation [J]. International Journal of Computer Assisted Radiology and Surgery, 2020, 15: 425-436. [25] Jin Logan. 3AU-Net: Triple attention U-Net for retinal vessel segmentation [C] //2020 IEEE 2nd International Conference on Civil Aviation Safety and Information Technology. Weihai: IEEE, 2020: 612-615. [26] Hu Jingfei, Wang Hua, Gao Shengbo, et al. S-Unet: A bridge-style U-Net framework with a saliency mechanism for retinal vessel segmentation [J]. IEEE Access, 2019, 7: 174167-174177. [27] Xiao Xiao, Lian Shen, Luo Zhiming, et al. Weighted Res-UNet for high-quality retina vessel segmentation [C]//2018 9th International Conference on Information Technology in Medicine and Education. Hangzhou: IEEE, 2018: 327-331. [28] 高宏杰, 邱天爽, 丑远婷,等. 基于改进U型网络的眼底图像血管分割 [J]. 中国生物医学工程学报, 2019, 38(1):1-8. [29] Hu Haigen, Zheng Yixing, Zhou Qianwei, et al. MC-Unet: multi-scale convolution Unet for bladder cancer cell segmentation in phase-contrast microscopy images [C]//2019 IEEE International Conference on Bioinformatics and Biomedicine. San Diego: IEEE, 2019: 1197-1199. [30] Zeng Zitao, Xie Weihao, Zhang Yunzhe, et al. RIC-Unet: an improved neural network based on Unet for nuclei segmentation in histology images [J]. IEEE Access, 2019, 7: 21420-21428. [31] Long Feixiao. Microscopy cell nuclei segmentation with enhanced U-Net [J]. BMC Bioinformatics, 2020, 21: 8. [32] Abraham N, Illanko K, Khan N, et al. Deep learning for semantic segmentation of brachial plexus nervesin ultrasound images using U-Net and M-Net[C]//2019 3rd International Conference on Imaging, Signal Processing and Communication. Singapore: IEEE, 2019: 85-89. [33] 吉彬, 任建君, 郑秀娟,等. 改进U-Net在喉白斑病灶分割中的应用 [J]. 计算机工程, 2020, 46(9): 248-253. [34] Huang Chenxi, Lan Yisha, Xu Gaowei, et al. A deep segmentation network of multi-scale feature fusion based on attention mechanism for IVOCT lumen contour [J]. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 2021, 18(1): 62-69. [35] Zhuang Zhemin, Li Nan, Joseph Raj AN, et al. An RDAU-NET model for lesion segmentation in breast ultrasound images [J]. PLoS ONE, 2019, 14(8): e0221535. [36] Rocha J, Cunha A, Mendonça AM. Conventional filtering versus U-net based models for pulmonary nodule segmentation in CT images [J]. Journal of Medical Systems, 2020, 44: 81. [37] Badrinarayanan V, Kendall A, Cipolla R. SegNet: a deep convolutional encoder-decoder architecture for image segmentation [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12): 2481-2495.
