The Similarity Evaluation Method of Wireless Capsule Endoscopy Images Based on ImprovedPatchMatch Algorithm
Tian Hao1,2, Lu Heng3, Pan Ning1,2*, Hu Huaifei1,2, Liu Haihua1,2
1(College of Biomedical Engineering, South-Central Minzu University, Wuhan 430074, China) 2(Hubei Key Laboratory of Medical Information Analysis and Tumor Diagnosis & Treatment, Wuhan 430074, China) 3(Department of Gastroenterology and Hepatology, Jinling Hospital, Medical School of Nanjing University, Nanjing 210002, China)
Abstract：The examination of every single wireless capsule endoscopy (WCE) produces a large quantity of image sequence data (i.e., 50 000~80 000 images), which creates big challenges in clinical practice and can lead to the difficulties in medical diagnosis. It is, therefore, of great significance to advance methods to quantify the abstract-level visual similarity between the WCE images and screen out redundant images in the WCE sequence, so that both the efficiency and the effectiveness of image diagnosis can be improved. In the current study, we proposed a PatchMatch algorithm based on the spatial constraint scheme to capture and evaluate the similarity between the WCE images. This novel method improved the conventional PatchMatch procedure through the achievement of matching the local image patches based on the spatial constraint measures, adding the offset position constraint, restricting the initial matching region, and providing different matching search regions for the image patch to be coordinated. Next, the image-level ensemble descriptor was built by combining the position and texture attribute information of matched image patches. Finally, the similarity of two WCE images was evaluated by a joint probability between ensemble descriptors. A recurring screening experiments of 10 cases of WCE image sequences consisting of more than 5000 WCE imageries were conducted by employing different descriptor operators (such as: SIFT and Hog). The results showed that the precision, recall and F-measure of experiments were 93.73 %, 95.44 % and 94.77 % respectively, indicating the proposed algorithm is effective in analyzing the similarity between the WCE images.
田昊, 陆恒, 潘宁, 胡怀飞, 刘海华. 基于改进PatchMatch算法的胶囊内镜图像相似度评估方法[J]. 中国生物医学工程学报, 2022, 41(6): 680-690.
Tian Hao, Lu Heng, Pan Ning, Hu Huaifei, Liu Haihua. The Similarity Evaluation Method of Wireless Capsule Endoscopy Images Based on ImprovedPatchMatch Algorithm. Chinese Journal of Biomedical Engineering, 2022, 41(6): 680-690.
[1] Fornaroli F, Gaiani F, Vincenzi F, et al. Applications of wireless capsule endoscopy in pediatric age: an update [J]. Acta Bio Medica Atenei Parmensis, 2018, 89(9): 40-46. [2] Spiceland CM, Lodhia N. Endoscopy in inflammatory bowel disease: role in diagnosis, management, and treatment [J]. World Journal of Gastroenterology, 2018, 24(35): 54-60. [3] 潘宁. 胶囊内镜图像序列冗余数据筛查方法研究 [D]. 武汉: 华中科技大学,2013. [4] Al-Shebani Q, Premaratne P, Mcandrew DJ, et al. A frame reduction system based on a color structural similarity (CSS) method and Bayer images analysis for capsule endoscopy [J]. Artificial Intelligence in Medicine, 2019, 94: 18-27. [5] 张林琪,郭旭东,张璐璐,等. 基于图像块信息相似性算法的胶囊内窥镜图像去冗余研究 [J]. 电子测量技术, 2020, 43(22): 93-97. [6] Hong Liu, Ning Pan, Heng Lu, et al. Wireless capsule endoscopy video reduction based on camera motion estimation [J]. Journal of Digital Imaging, 2013, 26(2): 287-301. [7] 彭同胜,刘小燕,龚军辉,等. 基于颜色匹配和改进LBP的胶囊内镜视频缩减 [J]. 电子测量与仪器学报, 2016, 30(9): 1379-1388. [8] Iakovidis DK, Tsevas S, Polydorou A. Reduction of capsule endoscopy reading times by unsupervised image mining [J]. Computerized Medical Imaging & Graphics, 2010, 34(6): 471-478. [9] 孙宇千,吕庆文,刘哲星,等. 胶囊内窥镜冗余图像数据自动筛除方法 [J]. 计算机应用研究, 2012, 29(6): 2393-2396. [10] 刘小燕,龚军辉,李向东,等. 基于SURF算法的胶囊内镜冗余图像快速筛除 [J]. 中国生物医学工程学报, 2016, 35(3): 264-271. [11] Biniaz A, Zoroofi RA, Sohrabi MR. Automatic reduction of wireless capsule endoscopy reviewing time based on factorization analysis [J]. Biomedical Signal Processing and Control, 2020, 59: 101897. [12] Barnes C, Shechtman E, Finkelstein A, et al. PatchMatch: a randomized correspondence algorithm for structural image editing [J]. ACM Trans Graph, 2009, 28(3): 24. [13] Besse F, Rother C, Fitzgibbon A, et al. PMBP: PatchMatch belief propagation for correspondence field estimation [J]. International Journal of Computer Vision, 2014, 110(1): 2-13. [14] 桑苗苗,彭进先,达通航,等. 基于PatchMatch的半全局高效双目立体匹配算法 [J]. 计算机科学, 2021, 48(1): 204-208. [15] 邓非,颜青松,肖腾. 多视密集匹配并行传播GPU-PatchMatch算法 [J]. 测绘学报, 2020, 49(02): 181-190. [16] Li Chunyuan, Hamza AB, Bouguila N, et al. Online redundant image elimination and its application to wireless capsule endoscopy [J]. Signal Image & Video Processing, 2014, 8(8): 1497-1506. [17] Boiman O, Irani M. Detecting irregularities in images and in video [J]. International Journal of Computer Vision, 2007, 74(1): 17-31. [18] Aslan MF, Durdu A, Sabanci K, et al. CNN and HOG based comparison study for complete occlusion handling in human tracking [J]. Measurement, 2020, 158: 107704. [19] Zhong Yuanchang, Sun Lili, Ge Chenhao, et al. HOG-ESRs face emotion recognition algorithm based on HOG feature and ESRs method [J]. Symmetry, 2021, 13(2): 228. [20] Sthevanie F, Ramadhani KN. Spoofing detection on facial images recognition using LBP and GLCM combination [J]. Journal of Physics Conference, 2018, 971: 012014. [21] Kaplan K, Kaya Y, Kuncan M, et al. Brain tumor classification using modified local binary patterns (LBP) feature extraction methods [J]. Medical Hypotheses, 2020, 139: 109696. [22] Parashivamurthy R, Naveena C, Sharath Kumar YH. SIFT and HOG features for the retrieval of ancient Kannada epigraphs [J]. IET Image Processing, 2020, 14(17): 4657-4662. [23] 江少锋,黄志鹏,杨素华,等. 基于SIFT特征关键点匹配的脑组织提取方法 [J]. 中国生物医学工程学报, 2018, 37(3): 266-273.
