Abstract Identifying useful concepts from large scale medical images is an important technology for image knowledge representation. Developing semantic concept detection algorithms is helpful to machine understanding and learning latent knowledge from medical images, and plays an important role in image-assisted diagnosis and intelligent image reading. In this study, the problem of detecting high-frequency concepts from medical images was transformed into a multi-label classification task. The deep transfer learning method based on convolutional neural network (CNNs) was used to recognize high-frequency medical concepts. The image retrieval-based topic modeling method was used to obtain the semantically related concepts from the similar images of given medical images. The CLEF cross language image retrieval track (ImageCLEF) launched the ImageCLEFcaption 2018 evaluation task on May, 2018, in which the Concept Detection subtask identified 111,156 semantic concepts from 222,314 training images and 9,938 test images. Our research group presented experimental results of both methods. The CNNs-based deep transfer learning method achieved the F1 score of 0.0928, which ranked second in all the submission teams. The retrieval-based topic model could recall some low-frequency concepts and achieved the F1 score of 0.0907, but dependent heavily on the image retrieval results. The results proved that the CNNs-based deep transfer learning method showed preferable robustness on high-frequency concept detection, but still a lot of room for improvement in the research of large-scale open semantic concept detection.
[1] 刘飞,张俊然,杨豪. 基于深度学习的医学图像识别研究进展[J]. 中国生物医学工程学报, 2018,37(1): 87-93. [2] Litjens G, Kooi T, Bejnordi BE, et al. A survey on deep learning in medical image analysis [J]. Medical Image Analysis, 2017, 42(9): 60-88. [3] He Kaiming, Zhang Xiangyu, Ren Shaoqing, et al. Deep residual learning for image recognition [C] // The IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778. [4] Koniusz P, Yan Fei, Gosselin P H, et al. Higher-order occurrence pooling for bags-of-words: visual concept detection [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(2): 313-326. [5] 李亚鹏,万遂人. 基于深度学习的行人属性多标签识别[J]. 中国生物医学工程学报, 2018, 37(4): 423-428. [6] 孔小函,檀韬,包凌云,等. 基于卷积神经网络和多信息融合的三维乳腺超声分类方法[J]. 中国生物医学工程学报, 2018, 37(4): 414-422. [7] 王翀,何兴鑫,方乐缘,等. 基于联合决策卷积神经网络的光学相干断层扫描图像自动分类[J]. 中国生物医学工程学报, 2018, 37(6): 641-648. [8] Pan Jialin, Yang Qiang. A survey on transfer learning [J]. IEEE Transactions on Knowledge and Data Engineering, 2010, 22(10): 1345-1359. [9] Esteva A, Kuprel B, Novoa RA, et al. Dermatologist-level classification of skin cancer with deep neural networks [J]. Nature, 2017, 542(7639): 115-118. [10] 于玉海,林鸿飞,孟佳娜,等. 跨模态多标签生物医学图像分类建模识别[J]. 中国图像图形学报, 2018, 23(6): 917-927. [11] ImageCLEF. ImageCLEFcaption 2017 Homepage. [EB/OL]. http://www.imageclef.org/2017/caption, 2017-09-14/2018-11-16. [12] ImageCLEF. ImageCLEFcaption 2018 Homepage. [EB/OL]. http://www.imageclef.org/2018/caption, 2018-09-14/2018-11-16 [13] NCBI. PubMed. [EB/OL]. https://www.ncbi.nlm.nih.gov/pmc/, 2019-2-26/2019-03-22. [14] U.S. National Library of Medicine. Unified Medical Language System (UMLS). [EB/OL]. https://www.nlm.nih.gov/research/umls/, 2018-10-11/2019-03-22. [15] Bodenreider O. The unified medical language system (UMLS): integrating biomedical terminology [J]. Nucleic Acids Research, 2004,32:267-270. [16] Eickhoff C, Schwall I, Seco de Herrera, et al. Overview of ImageCLEFcaption 2017 - image caption prediction and concept detection for biomedical images [C] //Lecture Notes in Computer Science: Experimental IR meets multilinguality, multimodality, and interaction. 8th International Conference of the CLEF Association (CLEF 2017). Dublin, Ireland. Cham: Springer, 2017: 315- 337. [17] Seco de Herrera A.G, Eickhoff C, Andrearczyk V, et al. Overview of the image CLEF 2018 caption prediction tasks[C]// ImageCLEF - Multimedia Retrieval in CLEF. Avignon, France: CEUR Workshop, 2018: 2125. [18] Pinho E, Figueira J, Silva JM, et al. Towards representation learning for biomedical concept detection in medical images: UA.PT bioinformatics in ImageCLEF 2017. [EB/OL]. http://120.52.51.13/ceur-ws.org/Vol-1866/paper_149.pdf. 2017-07-11/2019-03-22. [19] Dimititris K, Ergina K. Concept detection on medical images using deep residual learning network. [EB/OL]. http://120.52.51.13/ceur-ws.org/Vol-1866/paper_122.pdf. 2017-07-13/2019-03-22. [20] Valavanis L, Stathopoulos S. IPL at imageCLEF 2017 concept detection task. [EB/OL]. http://www.dei.unipd.it/～ferro/CLEF-WN-Drafts/CLEF2017/paper_144.pdf. 2017-07-11/2019-03-22. [21] Rahman M, Lagree T, Taylor M. A cross-modal concept detection and caption prediction approach in ImageCLEFcaption track of ImageCLEF 2017. [EB/OL]. http://59.80.44.98/ceur-ws.org/Vol-1866/paper_160.pdf. 2017-07-11/2019-03-22. [22] Ny Hoavy N, Mothe J, Randrianarivony MI. IRIT & MISA at image CLEF 2017-multi label classification. [EB/OL]. http://59.80.44.100/ceur-ws.org/Vol-1866/paper_148.pdf. 2017-07-11/2019-03-22. [23] Soldaini L, Goharian N. QuickUMLS: A fast, unsupervised approach for medical concept extraction. [EB/OL]. http://medir2016.imag.fr/data/MEDIR_2016_paper_16.pdf. 2016-07-27/2019-03-22. [24] Szegedy C, Vanhoucke V, Ioffe S, et al. Rethinking the inception architecture for computer vision[C]//The IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas: IEEE Computer Society, 2016:2818-2826. [25] Russakovsky O, Deng Jia, Su Hao, et al. Imagenet large scale visual recognition challenge [J]. International Journal of Computer Vision, 2015, 115(3): 211-252. [26] Mathias Lux. LIRE: Lucene Image Retrieval. [EB/OL]. http://www.lire-project.net/. 2015-8-21/2018-11-16 [27] Gan R, Yin J. Using LIRe to implement image retrieval system based on multi-feature descriptor[C]// The 3rd International Conference on Digital Manufacturing & Automation. Guilin: IEEE Computer Society, 2012: 1014-1017. [28] Blei DM, Ng AY, Jordan MI. Latent dirichlet allocation [J]. Journal of Machine Learning Research, 2003, 3:993-1022. [29] Pinho E, Costa C. Feature learning with adversarial networks for concept detection in medical images: UA.PT bioinformatics at ImageCLEF 2018[C]// ImageCLEF - Multimedia Retrieval in CLEF CEUR Workshop Proceedings. Avignon: CEUR-WS, 2018:10-14.
