Retinal Lesion Segmentation with Retinopathy Guidance Deterministic Representation from Diffusion Models
Xie Yingpeng, Qu Junlong, Xie Hai, Wang Tianfu*, Lei Baiying1*
Guangdong Key Laboratory for Biomedical Measurements and Ultrasound Imaging,Department of Biomedical Engineering, School of Medicine, Shenzhen University, Shenzhen 518060, Guangdong, China
Abstract：Acquiring a comprehensive segmentation map of retinal lesions is a crucial step in developing an automated, interpretable diagnostic tool for retinopathy. However, the inherent diversity and complexity of retinal lesions, coupled with the high cost of precise annotation, pose substantial challenges to traditional supervised learning approaches. Recent advances suggest that representation learning can mitigate the reliance on extensive annotated data by pre-training robust image representation models on large-scale unlabeled datasets. In this study, we introduced an innovative representation learning framework based on denoising Diffusion Probabilistic Models (DDPM), specifically tailored to capture the subtle and localized variations in medical imagery, thereby providing precise feature representations for the segmentation of retinal lesions. Utilizing unlabeled fundus images, our approach learnt the reverse process of Markov diffusion, establishing a foundation for extracting pixel-level representations. A retinal lesion grading classifier, informed by domain knowledge of retinopathy severity and lesion correlation, was implemented to guide the reverse diffusion process to enhance representations pertinent to lesions. The guided representations served as a repository of intrinsic semantic information, offering robust image representations for downstream retinal segmentation tasks. In experiments on multiple fundus image datasets, our method achieved average Dice coefficients of 0.872 for optic cup and 0.877 for optic disc segmentation with only 50 samples. For diabetic retinopathy lesions, it reached a Dice coefficient of 0.664, and for age-related macular degeneration lesions, 0.513, demonstrating diffusion-based representation’s generality and effectiveness across various complex retinal conditions.
[1] Wang Renzhen, Chen Benzhi, Meng Deyu, et al. Weakly supervised lesion detection from fundus images [J]. IEEE Transactions on Medical Imaging, 2018, 38(6): 1501-1512. [2] Li Tao, Bo Wang, Hu Chunyu, et al. Applications of deep learning in fundus images: a review [J]. Medical Image Analysis, 2021, 69: 101971. [3] Playout C, Duval R, Cheriet F. A novel weakly supervised multitask architecture for retinal lesions segmentation on fundus images [J]. IEEE Transactions on Medical Imaging, 2019, 38(10): 2434-2444. [4] Sivaswamy J, Krishnadas S, Chakravarty A, et al. A comprehensive retinal image dataset for the assessment of glaucoma from the optic nerve head analysis [J]. JSM Biomedical Imaging Data Papers, 2015, 2(1): 1004. [5] Cabrera DeBuc D, Somfai GM, Koller A. Retinal microvascular network alterations: potential biomarkers of cerebrovascular and neural diseases [J]. American Journal of Physiology-Heart Circulatory Physiology, 2017, 312(2): H201-H212. [6] Huang Shiqi, Li Jianan, Xiao Yuze, et al. RTNet: relation transformer network for diabetic retinopathy multi-lesion segmentation [J]. IEEE Transactions on Medical Imaging, 2022, 41(6): 1596-1607. [7] Zhou Yi, Wang Boyang, Huang Lei, et al. A benchmark for studying diabetic retinopathy: segmentation, grading, and transferability [J]. IEEE Transactions on Medical Imaging, 2020, 40(3): 818-828. [8] Guo Song, Li Tao, Kang Hong, et al. L-Seg: An end-to-end unified framework for multi-lesion segmentation of fundus images [J]. Neurocomputing, 2019, 349: 52-63. [9] Jin Cheng, Guo Zhengrui, Lin Yi, et al. Label-efficient deep learning in medical image analysis: Challenges and future directions [EB/OL]. https://arxiv.org/abs/2303.12484, 2023-12-20/2024-01-02. [10] Chen T, Kornblith S, Norouzi M, et al. A simple framework for contrastive learning of visual representations [C] // International Conference on Machine Learning. Virtual: PMLR, 2020: 1597-1607. [11] Wang Xinlong, Zhang Rufeng, Shen Chunhua, et al. Dense contrastive learning for self-supervised visual pre-training [C] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Virtual: IEEE, 2021: 3024-3033. [12] Donahue J and Simonyan K, Large scale adversarial representation learning [C] // Advances in Neural Information Processing Systems. Cambridge: MIT Press, 2019: 10542-10552. [13] Preechakul K, Chatthee N, Wizadwongsa S, et al. Diffusion autoencoders: toward a meaningful and decodable representation [C] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022: 10619-10629. [14] Goodfellow I, Pouget-Abadie J, Mirza M, et al. Generative adversarial nets [C] // Advances in Neural Information Processing Systems. Cambridge: MIT Press, 2014: 2672-2680. [15] Zhang Yuxuan, Ling Huan, Gao Jun, et al. Datasetgan: efficient labeled data factory with minimal human effort [C] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Virtual: IEEE, 2021: 10145-10155. [16] Galeev D, Sofiiuk K, Rukhovich D, et al. Learning high-resolution domain-specific representations with a gan generator [C] // Structural, Syntactic, and Statistical Pattern Recognition: Joint IAPR International Workshops. Padua: Springer, 2021: 108-118. [17] Li Daiqing, Ling Huan, Kim S W, et al. BigDatasetGAN: synthesizing ImageNet with pixel-wise annotations [C] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022: 21330-21340. [18] Rewatbowornwong P, Tritrong N, and Suwajanakorn S, Repurposing GANs for one-shot semantic part segmentation [J]. IEEE Transactions on Pattern Analysis Machine Intelligence, 2022, 45(4): 5114-5125. [19] Xia Weihao, Zhang Yulun, Yang Yujiu, et al. Gan inversion: a survey [J]. IEEE Transactions on Pattern Analysis Machine Intelligence, 2022, 45(3): 3121-3138. [20] Abdal R, Qin Y, Wonka P. Image2stylegan: how to embed images into the stylegan latent space? [C] // Proceedings of the IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 4432-4441. [21] Ho J, Jain A, and Abbeel P, Denoising diffusion probabilistic models [C] // Advances in Neural Information Processing Systems. Cambridge: MIT Press, 2020: 6840-6851. [22] Dhariwal P and Nichol A, Diffusion models beat gans on image synthesis [C] // Advances in Neural Information Processing Systems. Cambridge: MIT Press, 2021: 8780-8794. [23] Wu Junde, Fu Rao, Fang Huihui, et al. MedSegDiff: medical image segmentation with diffusion probabilistic model [C] // Medical Imaging with Deep Learning. Nashville: PMLR, 2023: 1623-1639. [24] Fu Yunguan, Li Yiwen, Saeed SU, et al. A Recycling training strategy for medical image segmentation with diffusion denoising models [J]. Machine Learning for Biomedical Imaging, 2023, 2: 507-546. [25] Baranchuk D, Rubachev I, Voynov A, et al. Label-efficient semantic segmentation with diffusion models [C] // International Conference on Learning Representations. Virtual:OpenReview, 2022: 1-15. [26] Song J, Meng C, Ermon S. Denoising diffusion implicit models [C] // International Conference on Learning Representations. Vienna:OpenReview, 2020: 1-20. [27] Wei Qijie, Li Xirong, Yu Weihong, et al. Learn to segment retinal lesions and beyond [C] // 2020 25th International Conference on Pattern Recognition. Milan: IEEE, 2021: 7403-7410. [28] Orlando JI, Fu H, Breda JB, et al. Refuge challenge: a unified framework for evaluating automated methods for glaucoma assessment from fundus photographs [J]. Medical Image Analysis, 2020, 59: 101570. [29] Fang Huihui, Li Fei, Fu Huazhu, et al. Adam challenge: detecting age-related macular degeneration from fundus images [J]. IEEE Transactions on Medical Imaging, 2022, 41(10): 2828-2847. [30] Cuadros J, Bresnick G, EyePACS: an adaptable telemedicine system for diabetic retinopathy screening [J]. Journal of Diabetes Science Technology, 2009, 3(3): 509-516. [31] Heusel M, Ramsauer H, Unterthiner T, et al. Gans trained by a two time-scale update rule converge to a local nash equilibrium [C] // Advances in Neural Information Processing Systems. Cambridge: MIT Press, 2017: 6629-6640. [32] Naeem M F, Oh S J, Uh Y, et al. Reliable fidelity and diversity metrics for generative models [C] // International Conference on Machine Learning. Virtual: PMLR, 2020: 7176-7185. [33] Kynkäänniemi T, Karras T, Laine S, et al. Improved precision and recall metric for assessing generative models [C] // Advances in Neural Information Processing Systems. Cambridge: MIT Press, 2019: 3927-3936. [34] Parmar G, Zhang R, and Zhu J-Y, On aliased resizing and surprising subtleties in gan evaluation [C] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022: 11410-11420. [35] Kang M, Shin J, and Park J, StudioGAN: a taxonomy and benchmark of GANs for image synthesis [J]. IEEE Transactions on Pattern Analysis Machine Intelligence, 2023, 45(12), 15725-15742. [36] Karras T, Aittala M, Hellsten J, et al. Training generative adversarial networks with limited data [C] // Advances in Neural Information Processing Systems. Cambridge: MIT Press, 2020: 12104-12114. [37] Wu Junde, Fu Rao, Fang Huihui, et al. Medical sam adapter: adapting segment anything model for medical image segmentation [EB/OL]. https://arxiv.org/abs/2304.12620, 2023-12-29/2024-01-02. [38] Xie Yingpeng, Wan Qiwei, Xie Hai, et al. Fundus image-label pairs synthesis and retinopathy screening via GANs with class-imbalanced semi-supervised Learning [J]. IEEE Transactions on Medical Imaging, 2023, 42(9): 2714-2725. [39] Lv Junyan, Zhang Yiqi, Huang Yijin, et al. Aadg: automatic augmentation for domain generalization on retinal image segmentation [J]. IEEE Transactions on Medical Imaging, 2022, 41(12): 3699-3711. [40] Zhu Ye, Wu Yu, Deng Zhiwei, et al. Unseen image synthesis with diffusion models [EB/OL]. https://arxiv.org/abs/2310.09213, 2023-10-13/2024-01-02.
