Multi-Category Intestinal Polyp Image Classification Network Based on Edge Prior Information
Li Sheng1, Cao Jing1, Ye Shufang2*, Dai Fei2, He Xiongxiong1
1(College of Information Engineering, Zhejiang University of Technology, Hangzhou 310012, China) 2(Center of Digestive Endoscopy, People′s Hospital of Lishui, Lishui 323020, Zhejiang, China)
Abstract：The classification of intestinal polyps can help endoscopists to assist in diagnosis and distinguish between high-risk polyps requiring immediate treatment and low-risk polyps that can be deferred. Existing polyp classification algorithms based on deep learning can′t distinguish the high degree of inter-class similarities images, and need to be improved for multi-category polyp classification task. In this paper, a multi-category polyp image classification network based on edge prior information was proposed, including edge detection stage, edge feature descriptor extraction stage and polyp classification stage. Firstly, at the skip connection layer in the edge detection stage, a reverse attention edge monitoring module was designed and embedded to better capture the details of polyp edge. Secondly, under the guidance of the prior knowledge of the endoscopist, the perimeter size was represented by counting the number of pixels on the edge of the polyp, and the concavity and convexity were used to represent the smoothness feature, so as to supplement the insufficiency of neural network feature extraction. Finally, the channel attention was inserted after DenseBlock4 of the classification network to adaptively capture discriminative features. The private dataset was consisted of 1 050 desensitized original images that are collected from the Digestive Endoscopy Center of Lishui People′s Hospital within the year 2018 to 2019. Five-fold cross-validation was conducted in the polyp four-category dataset constructed in this paper, and the overall accuracy reached 77.29%, which was 6.46% higher than the best results of existing algorithms. The classification network fused with edge prior information can effectively discriminate two groups of polyp images with high degree of inter-class similarities, namely non-adenomatous polyps and low-grade adenomatous polyps, high-grade adenomatous polyps and adenocarcinoma. The established network in this paper increased the robustness and improved classification performance, providing auxiliary opinions for doctor diagnosis under limited training dataset.
李胜, 曹婧, 叶淑芳, 代飞, 何熊熊. 边缘先验信息下的多类型肠道息肉图像分类网络[J]. 中国生物医学工程学报, 2022, 41(4): 431-442.
Li Sheng, Cao Jing, Ye Shufang, Dai Fei, He Xiongxiong. Multi-Category Intestinal Polyp Image Classification Network Based on Edge Prior Information. Chinese Journal of Biomedical Engineering, 2022, 41(4): 431-442.
[1] 李兆申, 金震东, 令狐恩强. 中国早期结直肠癌筛查流程专家共识意见 (2019, 上海)[J]. 中华健康管理学杂志, 2019, 13: 376-386. [2] Zhang Siwei, Sun Kexin, Zheng Rongshou, et al. Cancer incidence and mortality in China, 2015[J]. Journal of the National Cancer Center, 2021, 1: 2-11. [3] Zauber AG, Winawer SJ, O′Brien MJ, et al. Colonoscopic polypectomy and long-term prevention of colorectal-cancer deaths[J]. N Engl J Med, 2012, 366: 687-696. [4] Simon K. Colorectal cancer development and advances in screening[J]. Clinical Interventions in Aging, 2016, 11: 967-976. [5] Ladabaum U, Fioritto A, Mitani A, et al. Real-time optical biopsy of colon polyps with narrow band imaging in community practice does not yet meet key thresholds for clinical decisions[J]. Gastroenterology, 2013, 144: 81-91. [6] Maeda Y, Kudo S, Mori Y, et al. Fully automated diagnostic system with artificial intelligence using endocytoscopy to identify the presence of histologic inflammation associated with ulcerative colitis (with video)[J]. Gastrointestinal Endoscopy, 2019, 89: 408-415. [7] IJspeert JEG, Bastiaansen BAJ, Van Leerdam ME, et al. Development and validation of the WASP classification system for optical diagnosis of adenomas, hyperplastic polyps and sessile serrated adenomas/polyps[J]. Gut, 2016, 65: 963-970. [8] Sano Y, Tanaka S, Kudo S, et al. Narrow‐band imaging (NBI) magnifying endoscopic classification of colorectal tumors proposed by the Japan NBI Expert Team[J]. Digestive Endoscopy, 2016, 28: 526-533. [9] Häfner M, Gangl A, Liedlgruber M, et al. Endoscopic image classification using edge-based features[C]//2010 20th International Conference on Pattern Recognition. Istanbul: IEEE, 2010: 2724-2727. [10] Tamaki T, Yoshimuta J, Kawakami M, et al. Computer-aided colorectal tumor classification in NBI endoscopy using local features[J]. Medical Image Analysis, 2013, 17: 78-100. [11] Mesejo P, Pizarro D, Abergel A, et al. Computer-aided classification of gastrointestinal lesions in regular colonoscopy[J]. IEEE Transactions on Medical Imaging, 2016, 35: 2051-2063. [12] 杨建军, 常丽萍, 李胜, 等. 基于新型特征和特征袋模型的内窥镜大肠病变辅助诊断[J]. 中国生物医学工程学报, 2020, 39: 404-412. [13] Zhang Ruikai, Zheng Yali, Mak TWC, et al. Automatic detection and classification of colorectal polyps by transferring low-level CNN features from nonmedical domain[J]. IEEE Journal of Biomedical and Health Informatics, 2016, 21: 41-47. [14] Byrne MF, Chapados N, Soudan F, et al. Real-time differentiation of adenomatous and hyperplastic diminutive colorectal polyps during analysis of unaltered videos of standard colonoscopy using a deep learning model[J]. Gut, 2019, 68: 94-100. [15] 黄佩云, 王凯妮, 何晓璞, 等. 基于卷积神经网络的肠息肉分类方法的初步研究[J]. 胃肠病学和肝病学杂志, 2021, 30(3): 336-340. [16] 潘燕七, 陈睿, 张旭, 等. 基于浅层与深层特征融合的胃癌前疾病识别[J]. 中国生物医学工程学报, 2020, 39: 413-421. [17] Wang Yi, Tavanapong W, Wong JS, et al. Detection of quality visualization of appendiceal orifices using local edge cross-section profile features and near pause detection[J]. IEEE Transactions on Biomedical Engineering, 2009, 57: 685-695. [18] Wang Yi, Tavanapong W, Wong J, et al. Part-based multiderivative edge cross-sectional profiles for polyp detection in colonoscopy[J]. IEEE Journal of Biomedical and Health Informatics, 2013, 18: 1379-1389. [19] Tajbakhsh N, Gurudu SR, Liang Jianming. Automated polyp detection in colonoscopy videos using shape and context information[J]. IEEE Transactions on Medical Imaging, 2015, 35: 630-644. [20] Hwang S, Oh JH, Tavanapong W, et al. Polyp detection in colonoscopy video using elliptical shape feature[C]//2007 IEEE International Conference on Image Processing. San Antonio: IEEE, 2007: 465-468. [21] Fan Dengping, Ji Gepeng, Zhou Tao, et al. Pranet: Parallel reverse attention network for polyp segmentation[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention. Oxford: Springer, Cham, 2020: 263-273. [22] He Kaiming, Zhang Xiangyu, Ren Shaoqing, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778. [23] Huang Gao, Liu Zhuang, Van Der Maaten L, et al. Densely connected convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: 2017: 4700-4708. [24] Guo Xiaoqing, Yuan Yixuan. Triple ANet: Adaptive abnormal-aware attention network for WCE image classification[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention. Shenzhen: Springer, Cham, 2019: 293-301. [25] Xing Xiaohan, Yuan Yixuan, Meng MQH. Zoom in lesions for better diagnosis: Attention guided deformation network for wce image classification[J]. IEEE Transactions on Medical Imaging, 2020, 39: 4047-4059. [26] Guo Xiaoqing, Yuan Yixuan. Semi-supervised WCE image classification with adaptive aggregated attention[J]. Medical Image Analysis, 2020, 64: 1-13. [27] Niu Zhaoyang, Zhong Guoqiang, Yu Hui. A review on the attention mechanism of deep learning[J]. Neurocomputing, 2021, 452: 48-62. [28] Tanaka S, Oka S, Hirata M, et al. Pit pattern diagnosis for colorectal neoplasia using narrow band imaging magnification[J]. Digestive Endoscopy, 2006, 18(Suppl. 1)): S52-S56. [29] Buda M, Maki A, Mazurowski MA. A systematic study of the class imbalance problem in convolutional neural networks[J]. Neural Networks, 2018, 106: 249-259. [30] Soffer S, Klang E, Shimon O, et al. Deep learning for wireless capsule endoscopy: a systematic review and meta-analysis[J]. Gastrointestinal Endoscopy, 2020, 92(4): 831-839. [31] Hu Jie, Shen Li, Sun Gang. Squeeze-and-excitation networks[C]//Proceedings of the IEEE Conference On Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 7132-7141. [32] Guan Qingji, Huang Yaping, Zhong Zhun, et al. Thorax disease classification with attention guided convolutional neural network[J]. Pattern Recognition Letters, 2020, 131: 38-45. [33] Zhu Meilu, Chen Zhen, Yuan Yixuan. DSI-Net: Deep Synergistic Interaction Network for Joint Classification and Segmentation with Endoscope Images[J]. IEEE Transactions on Medical Imaging, 2021, 40: 3315-3325. [34] 雍和礼, 张丽. 内镜下治疗结肠息肉的安全性及有效性[J]. 世界华人消化杂志, 2015, 23: 4585-4588. [35] Fu JJC, Yu Yawen, Lin Hongmau, et al. Feature extraction and pattern classification of colorectal polyps in colonoscopic imaging[J]. Computerized Medical Imaging and Graphics, 2014, 38: 267-275. [36] Rodriguez-Diaz E, Baffy G, Lo WK, et al. Real-time artificial intelligence-based histologic classification of colorectal polyps with augmented visualization[J]. Gastrointestinal Endoscopy, 2021, 93: 662-670. [37] 于俊清, 吴泽斌, 吴飞, 等. 多媒体工程: 2016—图像检索研究进展与发展趋势[J]. 中国图像图形学报, 2017, 22: 1467-1485.
