|
|
|
| Glandular Segmentation Based on Shape Stream and Multi-Scale Feature Fusion |
| Lin Jiawen1,2,3, Chen Susu1,2,3, Lin Zhiming4, Li Li5, Weng Qian1,2,3* |
1(College of Computer and Data Science, Fuzhou University, Fuzhou 350108, China)
2(Big Data Intelligence Engineering Research Center of the Ministry of Education, Fuzhou University, Fuzhou 350108, China)
3(Fujian Key Laboratory of Network Computing and Intelligent Information Processing, Fuzhou University, Fuzhou 350108, China)
4(Ruijie Network Co., Ltd., Fuzhou 350007, China)
5(Department of Ophthalmology, Fuzhou University Affiliated Provincial Hospital, Fuzhou 350001,China) |
|
|
|
|
Abstract Meibomian gland imaging technology is widely used in the classification diagnosis, management and personalized treatment of dry eye syndrome. Direct observation and qualitative evaluation by ophthalmologists may result in low subjective and reproducible evaluation. To improve the diagnostic efficiency of ophthalmologists, researchers have proposed a series of gland segmentation method for infrared meibomian gland images based on U-Net. However, the segmentation results are still not ideal at image edges, at locations of reflective points, and in areas with dense glandular structures. Considering the characteristics of infrared meibomian gland imaging and glandular distribution, this paper proposed a glandular segmentation model SS-UNet based on shape stream and multi-scale feature fusion, introduced an atrous convolution module to enhance the model′s feature extraction ability, designed a shape stream auxiliary branch to fully learn the shape information of glands, used a multi-scale feature fusion module to obtain feature representations of glands with different thicknesses. To verify the validity of the model, a fully annotated dataset containing 203 infrared meibormian gland images were collected by the ophthalmology department of Fujian Provincial Hospital and used to conduct comparative experiments with other advanced medical segmentation models in the same experimental environment, perform module ablation analysis, and display the visualization results. The experimental results showed that the Acc, Dice, IoU indicators of SS-UNet reached 94.62%, 80.94%, and 68.17%, respectively, which were improved by 0.36%, 1.41%, and 1.95% compared to the benchmark network U-Net, significantly improving the gland segmentation results. This work has shown that SS-UNet was able to fully utilize information such as the shape and scale of glands to solve incorrect segmentation problems such as glandular adhesions and missed detections, effectively improving segmentation accuracy and providing objective basis for assisting clinical diagnosis.
|
|
Received: 16 January 2024
|
|
|
|
Corresponding Authors:
*E-mail: fzuwq@fzu.edu.cn
|
|
|
|
[1] 亚洲干眼协会中国分会, 海峡两岸医药卫生交流协会眼科学专业委员会眼表与泪液病学组, 中国医师协会眼科医师分会眼表与干眼学组. 中国干眼专家共识:定义和分类 (2020年)[J]. 中华眼科杂志, 2020, 56(6): 418-422.
[2] 马天行, 袁媛, 唐鸥. 干眼症中西医研究进展综述[J]. 世界最新医学信息文摘, 2020, 20(27): 61-62.
[3] 王淑兰. 干眼症研究进展的综述[J]. 世界最新医学信息文摘, 2018, 18(11): 32-33.
[4] Song Peige, Wei Xia, Wang Manli, et al. Variations of dry eye disease prevalence by age, sex and geographic characteristics in China: a systematic review and meta-analysis[J]. Journal of Global Health, 2018, 8(2):020503.
[5] Chung HS, Han YE, Lee H, et al. Intense pulsed light treatment of the upper and lower eyelids in patients with moderate-to-severe meibomian gland dysfunction[J]. International Ophthalmology, 2023, 43(1):73-82.
[6] Huang Binge, Fei Fangrong, Wen Han, et al. Impacts of gender and age on meibomian gland in aged people using artificial intelligence[J]. Frontiers in Cell and Developmental Biology, 2023,11:1199440.
[7] Suzuki T, Teramukai S, Kinoshita S. Meibomian glands and ocular surface inflammation[J]. The Ocular Surface, 2015, 13(2): 133-149.
[8] Nelson JD, Shimazaki J, Benitez-del-Castillo JM, et al. The international workshop on meibomian gland dysfunction: report of the definition and classification subcommittee[J]. Investigative Ophthalmology & Visual Science, 2011, 52(4): 1930-1937.
[9] Fineide F, Arita R, Utheim TP. The role of meibography in ocular surface diagnostics: A review[J]. The Ocular Surface, 2021, 19: 133-144.
[10] 亚洲干眼协会中国分会, 海峡两岸医药卫生交流协会眼科学专业委员会眼表与泪液病学组, 中国医师协会眼科医师分会眼表与干眼学组. 中国干眼专家共识:检查和诊断 (2020年)[J]. 中华眼科杂志, 2020, 56(10): 741-747.
[11] Pult H, Riede-Pult BH. Non-contact meibography: Keep it simple but effective[J]. Contact Lens and Anterior Eye, 2012, 35(2): 77-80.
[12] Tomlinson A, Bron AJ, Korb DR, et al. The international workshop on meibomian gland dysfunction: report of the diagnosis subcommittee[J]. Investigative Ophthalmology & Visual Science, 2011, 52(4): 2006-2049.
[13] Arita R, Suehiro J, Haraguchi T, et al. Objective image analysis of the meibomian gland area[J]. British Journal of Ophthalmology, 2014, 98(6): 746-755.
[14] Xiao Peng, Luo Zhongzhou, Deng Yuqing, et al. An automated and multiparametric algorithm for objective analysis of meibography images[J]. Quantitative Imaging in Medicine and Surgery, 2021, 11(4): 1586-1599.
[15] 骆仲舟, 邓宇晴, 王耿媛, 等. 基于红外成像原理的睑板腺图像量化分析系统[J]. 眼科学报, 2021, 36(1): 30-37.
[16] Koprowski R, Wilczyński S, Olczyk P, et al. A quantitative method for assessing the quality of meibomian glands[J]. Computers in Biology and Medicine, 2016, 75: 130-138.
[17] 薛连剑. 睑板腺近红外图像处理算法的研究[D]. 上海:上海交通大学, 2016.
[18] 左应波. 睑板腺图像处理与分析的研究[D]. 厦门:厦门大学, 2016.
[19] Maskin SL, Testa WR. Infrared video meibography of lower lid meibomian glands shows easily distorted glands: implications for longitudinal assessment of atrophy or growth using lower lid meibography[J]. Cornea, 2018, 37(10): 1279-1286.
[20] Chen Jian, Wan Jiaze, Fang Zhenghan, et al. LMSA-Net: a lightweight multi-scale aware network for retinal vessel segmentation [J]. International Journal of Imaging Systems and Technology, 2023,33(5): 1515-1530.
[21] Liu Yanhong, Shen Ji, Yang Lei, et al. ResDO-UNet: a deep residual network for accurate retinal vessel segmentation from fundus images[J]. Biomedical Signal Processing and Control, 2023, 79: 104087.
[22] Zhu Zhiqin, He Xianyu, Qi Guangqiu, et al. Brain tumor segmentation based on the fusion of deep semantics and edge information in multimodal MRI[J]. Information Fusion, 2023, 91: 376-387.
[23] Chen Jian, Lu Ranlin, Ye Shilin, et al. Image recovery matters: a recovery-extraction framework for robust fetal brain extraction from MR Images[J]. IEEE Journal of Biomedical and Health Informatics,23 Nov,2023[Epub ahead of print].
[24] Dong Caixia, Dai Duwei, Zhang Yizhi, et al. Learning from dermoscopic images in association with clinical metadata for skin lesion segmentation and classification[J]. Computers in Biology and Medicine, 2023, 152: 106321.
[25] Ronneberger O, Fischer P, Brox T. U-Net: convolutional networks for biomedical image segmentation[C]//Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI). Munich: Springer, Cham, 2015: 234-241.
[26] Li Li, Lai Taichen, Zou Jing, et al. Effects of orthokeratology lenses on tear film and tarsal glands and control of unilateral myopia in children[J]. Frontiers in Cell and Developmental Biology, 2023,11:1197262.
[27] Prabhu SM, Chakiat A, Shashank S, et al. Deep learning segmentation and quantification of Meibomian glands[J]. Biomedical Signal Processing and Control, 2020, 57(3): 101776.
[28] Dai Qi, Liu Xinyi, Lin Xiaolei, et al. A novel meibomian gland morphology analytic system based on a convolutional neural network[J]. IEEE Access, 2021, 9: 23083-23094.
[29] Wang Jiayun, Li Shixuan, Yeh TN, et al. Quantifying meibomian gland morphology using artificial intelligence[J]. Optometry and Vision Science, 2021, 98(9): 1094-1103.
[30] Khan ZK, Umar AI, Shirazi SH, et al. Image based analysis of meibomian gland dysfunction using conditional generative adversarial neural network[J]. BMJ Open Ophthalmology, 2021, 6(1): e436.
[31] Lin Zhiming, Lin Jiawen, Li Li. Automatic segmentation of glands in infrared meibomian gland image[C] //Proceedings of the International Conference on Information Technology in Medicine and Education (ITME). Fujian: IEEE, 2021: 399-403.
[32] Setu MAK, Horstmann J, Schmidt S, et al. Deep learning-based automatic meibomian gland segmentation and morphology assessment in infrared meibography[J]. Scientific Reports, 2021, 11:7649.
[33] Saha RK, Chowdhury AMM, Na KS, et al. AI-based automated Meibomian gland segmentation, classification and reflection correction in infrared Meibography [J/OL]. https://arxiv.org/abs/2205.15543.html,2022-5-31/2023-1-13.
[34] Zhang Zuhui., Lin Xiaolei., Yu Xinxin., et al. Meibomian gland density: an effective evaluation index of meibomian gland dysfunction based on deep learning and transfer learning[J]. Journal of Clinical Medicine,2022,11 (9): 2396.
[35] Goodfellow I, Pouget-Abadie J, Mirza M, et al. Generative adversarial nets[J]. Advances in Neural Information Processing Systems, 2014, 27:2672-2680.
[36] Liu Xiaoming, Wang Shuo, Zhang Ying, et al. Scribble-supervised meibomian glands segmentation in infrared images[J]. ACM Transactions on Multimedia Computing Communications and Applications, 2022,18(3):88.
[37] Szegedy C, Ioffe S, Vanhoucke V, et al. Inception-v4, Inception-ResNet and the impact of residual connections on learning[C]//Proceedings of the AAAI Conference on Artificial Intelligence (AAAI). San Francisco: AAAI Press, 2017:4278-4284.
[38] Yu F. Multi-scale context aggregation by dilated convolutions [J/OL]. https://arxiv.org/abs/1511.07122.html,2016-04-30/2023-1-13.
[39] He Kaiming, Zhang Xiangyu, Ren Shaoqing, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas: IEEE, 2016: 770-778.
[40] Lin T, Doll ARP, Girshick R, et al. Feature pyramid networks for object detection[C] //Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu: IEEE, 2017: 2117-2125.
[41] Woo S, Park J, Lee J, et al. CBAM: Convolutional block attention module[C]//Proceedings of the European Conference on Computer Vision (ECCV). Munich: Springer, Cham, 2018: 3-19.
[42] Zhou Zongwei, Siddiquee MMR, Tajbakhsh N, et al. UNet++: redesigning skip connections to exploit multiscale features in image segmentation[J]. IEEE Transactions on Medical Imaging, 2019, 39(6): 1856-1867.
[43] Huang Huiming, Lin Lanfen, Tong Ruofeng, et al. UNet 3+: a full-scale connected unet for medical image segmentation[C]//Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Barcelona: IEEE, 2020: 1055-1059.
[44] Oktay O, Schlemper J, Folgoc LL, et al. Attention U-Net: Learning Where to Look for the Pancreas. [J/OL]. https://arxiv.org/abs/1804.03999.html, 2018-05-20/2023-1-13.
[45] Gu Ran, Wang Guotai, Song Tao, et al. CA-Net: Comprehensive attention convolutional neural networks for explainable medical image segmentation[J]. IEEE Transactions on Medical Imaging, 2020, 40(2): 699-711.
[46] Fu Huazhu, Cheng Jun, Xu Yanwu, et al. Joint optic disc and cup segmentation based on multi-label deep network and polar transformation[J]. IEEE Transactions on Medical Imaging, 2018, 37(7): 1597-1605.
[47] Zhang Shihao, Fu Huazhu, Yan Yuguang, et al. Attention guided network for retinal image segmentation[C] //Proceedings of the International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI). Shenzhen: Springer, Cham, 2019: 797-805.
[48] Cao Hu, Wang Yueyue, Chen Joy, et al. Swin-Unet: Unet-like pure transformer for medical image segmentation[C] //Proceedings of the European Conference on Computer Vision (ECCV). Tel Aviv: Springer, Cham, 2022: 205-218.
[49] Chen Liang-chieh, Zhu Yukun, Papandreou G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//Proceedings of the European Conference on Computer Vision (ECCV). Munich: Springer, Cham, 2018: 801-818.
[50] Liu Dengfeng, Zhu Weifang, Zhuang Xinyu, et al. IF-Net: information fusion network for meibomian gland area and atrophy area segmentation[C]//Medical Imaging 2023: Image Processing. San Diego:SPIE, 2023, 12464: 917-924.
[51] Ibtehaz N, Rahman MS. MultiResUNet: Rethinking the U-Net architecture for multimodal biomedical image segmentation[J]. Neural Networks, 2020, 121: 74-87.
[52] Zheng Shaohua, Nie Weiyu, Pan Lin, et al. A dual-attention V-network for pulmonary lobe segmentation in CT scans[J]. IET Image Processing, 2021, 15(8): 1644-1654.
[53] 吴港,全海燕. 一种基于特征融合的息肉分割双解码模型[J]. 数据采集与处理, 2024, 39(4):954-966.
[54] Elnakib A, Gimel′farb G, Suri JS, et al. Medical image segmentation: a brief survey[J]. Multi Modality State-of-the-Art Medical Image Segmentation and Registration Methodologies, 2011(2): 1-39.
[55] Yao Dongren, Sui Jing, Wang Mingliang, et al. A mutual multi-scale triplet graph convolutional network for classification of brain disorders using functional or structural connectivity[J]. IEEE Transactions on Medical Imaging, 2021, 40(4): 1279-1289. |
|
|
|
