Recognition of Esophageal Cancer and Precancerous Lesions Based on CNN and ViT Complementary Learning Network
Chen Tuo1, Lin Zhigang2*, Chen Hong2, Wu Nengguang2, Gao Yang1
1(Department of Informatization and Artificial Intelligence, Zhejiang Provincial People's Hospital (People's Hospital of Hangzhou Medical College), Hangzhou 314408,China) 2(Information Center, The First Affiliated Hospital of Fujian Medical University, Fuzhou 350005, China) 3(Department of Pathology, The First Affiliated Hospital of Fujian Medical University, Fuzhou 350005, China)
Abstract：Accurate identification of esophageal cancer and precancerous lesions can improve the success rate of early intervention and increase patients' survival expectations. However, the lesion area in esophageal endoscopic images has the problem of high intra class diversity and high inter class similarity, which makes it difficult for existing methods to extract effective features and affects recognition performance. Therefore, a classification method for esophageal cancer and precancerous lesions based on a complementary learning network of convolutional neural network (CNN) andvision Transformer (ViT) was proposed to improve recognition accuracy. This network consists of two parallel feature extraction branches, which are used to learn local and global features in the image. First, the local feature extraction branch based on multidimensional attention module can obtain more discriminative local detail features within the lesion area. Second, the global feature extraction branch based on multi-directional attention module is used to learn multi-scale global semantic information. Finally, the cross scale complementary learning module is used to promote complementary learning between branches, improve the feature expression ability of the entire network, and achieve high-precision recognition of the disease. Experimental validation was conducted on a dataset of 3 730 white light endoscopic images of esophageal cancer and precancerous lesions, and the recognition accuracy reached 96.2%, exceeding the baseline model by 5.6 percentage points and outperforming other methods compared in the experiment; The optimal recognition accuracy was also achieved for each category on a public dataset of 6 000 gastrointestinal diseases(Kvasir-dataset), demonstrating good generalization ability. The proposed recognition model based on CNN and ViT complementary learning network can better capture the rich visual features in esophageal endoscopic images, thereby effectively improving recognition accuracy and providing important value for doctor assisted diagnosis.
陈拓, 林志刚, 陈虹, 吴能光, 高扬. 基于卷积神经网络和ViT互补学习网络的食管癌及癌前病变识别[J]. 中国生物医学工程学报, 2026, 45(2): 167-177.
Chen Tuo, Lin Zhigang, Chen Hong, Wu Nengguang, Gao Yang. Recognition of Esophageal Cancer and Precancerous Lesions Based on CNN and ViT Complementary Learning Network. Chinese Journal of Biomedical Engineering, 2026, 45(2): 167-177.
[1] Sung H, Ferlay J, Siegel RL, et al. Global Cancer Statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries[J].CA: A Cancer Journal for Clinicians, 2021,71(3):209-249. [2] Chen Xixi, Zhong Qian, Liu Yang, et al. Genomic comparison of esophageal squamous cell carcinoma and its precursor lesions by multi-region whole-exome sequencing[J]. Nature Communications,2017,8(1):524. [3] Naveed M, Kubiliun N. Endoscopic treatment of early-stage esophageal cancer[J]. Current Oncology Reports, 2018, 20(9): 71. [4] 赫捷,陈万青,李兆申,等.中国食管癌筛查与早诊早治指南(2022,北京)[J].中国肿瘤,2022,31(6):401-436. [5] Kim SH, Hong SJ. Current status of image-enhanced endoscopy for early identification of esophageal neoplasms[J].Clinical Endoscopy 2021,54(4):464-476. [6] Iwatsubo T, Ishihara R, Yamasaki Y, et al. Narrow band imaging under less-air condition improves the visibility of superficial esophageal squamous cell carcinoma[J]. BMC Gastroenterology,2020,20(1):389. [7] Li Bing, Cai Shilun, Tan Weimin, et al. Comparative study on artificial intelligence systems for detecting early esophageal squamous cell carcinoma between narrow-band and white-light imaging[J].World Journal of Gastroenterology,2021,27(3):281-293. [8] 王士旭, 柯岩, 刘雨蒙, 等. 内镜下早期食管癌及癌前病变识别人工智能YOLOv51模型的建立及临床验证[J]. 中华肿瘤杂志, 2022, 44(5):395-401. [9] Horie Y, Yoshio T, Aoyama K,et al.Diagnostic outcomes of esophageal cancer by artificial intelligence using convolutional neural networks[J].Gastrointestinal Endoscopy, 2019,89(1):25-32. [10] Feng Yadong, Liang Yan, Li Peng, et al. Artificial intelligence assisted detection of superficial esophageal squamous cell carcinoma in white-light endoscopic images by using a generalized system[J]. Discover Oncology,2023,14(1):73. [11] Tan Jinlin, Chinnaratha MA, Woodman R, et al. Diagnostic accuracy of artificial intelligence (AI) to detect early neoplasia in barrett's esophagus: a non-comparative systematic review and meta-analysis[J]. Frontiers in Medicine,22 June,2022[Epub ahead of print]. [12] Liu Wei, Yuan Xianglei, Guo Linjie, et al. Artificial intelligence for detecting and delineating margins of early ESCC under WLI endoscopy[J]. Clinical and Translational Gastroenterology, 2022, 13(1): e00433. [13] Vaswani A , Shazeer N , Parmar N ,et al.Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. New Yourk:ACM,2017:6000-6010. [14] Dosovitskiy A , Beyer L , Kolesnikov A ,et al.An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale.[EB/OL].https://arxiv.org/abs/2010.11929, 2021-06-03/2024-10-01. [15] Chen Hanting,Wang Yunhe,Guo Tianyu,et al.Pre-Trained Image Processing Transformer[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).Virtual Conference:IEEE,2021: 12299-12310. [16] Guo Jianyuan,Han Kai,Wu Han,et al. Cmt: Convolutional n eural networks meet vision transformers[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).New Orleans:IEEE,2022:12175-12185. [17] Han Kai,Wang Yunhe,Chen Hanting,et al. A survey on vision transformer[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2022,45(1):87-110. [18] Dai Yin, Gao Yifan, Liu Fayu. Transmed: transformers advance multi-model medical image classification[J]. Diagnostics,31 July,2021[Epub ahead of print]. [19] Leng Bing,Wang Chunqing,Leng Min,et al. Deep learning detection network for peripheral blood leukocytes based on improved detection transformer[J]. Biomedical Signal Processing and Control,2023,82(1):104518. [20] 柳懿垚,杨意,陈敏思,等.基于双分支多视角深度自注意力编码器的乳腺肿瘤分类方法[J].中国生物医学工程学报,2022,41(5):527-536. [21] 王润媛,陈星材,吴蔚,等.基于增强CT图像和Swin Transformer网络的食管癌T分期智能诊断模型的构建与评估[J].陆军军医大学学报,2023,45(16):1770-1778. [22] He Kaiming,Zhang Xiangyu,Ren Shaoqing,et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).Las Vegas:IEEE,2016:770-778. [23] 陈拓,石浩,李翔杰,等.基于时空特征自适应融合网络的加密流量分类方法[J].中南民族大学学报(自然科学版),2024,43(3):384-392. [24] Deng Jia,Dong Wei,Socher R,et al.Imagenet:a large-scale hierarchical image database[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Miami:IEEE,2009:248-255. [25] Du Wenju,Rao Nini,Dong Changlong,et al.Automatic classification of esophageal disease in gastroscopic images using an efficient channel attention deep dense convolutional neural network[J].Biomedical Optics Express,2021,12(6):3066-3081. [26] Liu Gaoshuang,Hua Jie,Wu Zhan,et al.Automatic classification of esophageal lesions in endoscopic images using a convolutional neural network[J].Annals of Translational Medicine,2020,8(7):486. [27] Wang Qilong,Wu Banggu,Zhu Pengfei,et al. ECA-Net: efficient channel attention for deep convolutional neural networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).Washington:IEEE,2020: 11531-11539. [28] 龙其刚,王金铭,梁燕,等.基于卷积神经网络的内镜图像中食管病变分类[J].计算机工程与应用,2023,59(7):118-125. [29] Pogorelov K,Randel K,Griwodz C,et al.Kvasir:a multi-class image dataset for computer aided gastrointestinal disease detection[C]//Proceedings of the 8th ACM on Multimedia Systems Conference.Taipei:ACM,2017:164-169. [30] Li Sheng,Cao Jing,Yao Jiafeng,et al.Adaptive aggregation with self-attention network for gastrointestinal image classification[J].IET Image Processing,2022,16(9):2384-2397. [31] 周涛,刘赟璨,侯森宝,等.M3 Res-Transformer:新冠肺炎胸部X-ray图像识别模型[J].电子学报,2024,52(2):589-601. [32] 林云浩,王艳军,李少春,等.一种耦合DeepLab与Transformer的农作物种植类型遥感精细分类方法[J].测绘学报,2024,53(2):353-366. [33] Graham B, El-Nouby A, Touvron H,et al. Levit: a vision transformer in convnet's clothing for faster inference[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision(CVPR). Virtual Conference:IEEE,2021: 12259-12269. [34] Dai Zihang,Liu Hanxiao,Tan Mingxing,et al. Coatnet: Marrying convolution and attention for all data sizes[J]. Advances in Neural Information Processing Systems, 2021,34:3965-3977. [35] 李军,叶欣怡,杨长才,等.基于联合深度网络和形态结构约束的三维医学图像分割方法[J].中国生物医学工程学报,2023,42(1):30-40. [36] Zhao Zixiang,Bai Haowen,Zhang Jiangshe, et al. Cddfuse: Correlation-driven dual-branch feature decomposition for multimodality image fusion[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).Vancouver:IEEE,2023: 5906-5916. [37] 杨芳,邹迎,丁雪妍,等.多层级深度特征融合的乳腺癌病理图像分类[J].光电子·激光,2025,36:185-192. [38] Chen Yinpeng,Dai Xiyang,Chen Dongdong,et al. Mobile-former: bridging mobilenet and transformer[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).New Orleans:IEEE,2022:5270-5279.
