医学图像语义概念识别方法研究

doi:10.3969/j.issn.0258-8021.2019.03.007

摘要
图/表
参考文献
相关文章 (15)

全文: PDF (2728 KB) HTML (1 KB)
输出: BibTeX | EndNote (RIS)

摘要医学图像语义概念识别是医学图像知识表示的重要技术环节。研究医学图像语义概念识别方法,有助于机器理解和学习医学图像中的潜在医学知识,在影像辅助诊断和智能读片等应用中发挥重要作用。将医学图像的高频概念识别问题转化为多标签分类任务,利用基于卷积神经网络的深度迁移学习方法,识别有限数量的高频医学概念;同时利用基于图像检索的主题建模方法,从给定医学图像的相似图像中提取语义相关概念。国际跨语言图像检索论坛ImageCLEF于2018年5月组织ImageCLEFcaption 2018评测,其子任务“概念检测”的目标是给定222 314张训练图片和9 938张测试图片,识别111 156个语义概念。上述两种方法的实验结果已被提交。实验结果表明,利用基于卷积神经网络的深度迁移学习方法识别医学图像高频概念,F1值为0.092 8,在提交团队中排名第二;基于图像检索的主题模型可召回部分低频相关概念,F1值为0.090 7,然而其性能依赖于图像检索结果的质量。基于卷积神经网络的深度迁移学习方法识别医学图像高频概念的鲁棒性优于基于图像检索方法的鲁棒性,但在大规模开放语义概念的识别技术研究上仍需进一步完善。

	服务

	把本文推荐给朋友
	加入我的书架
	加入引用管理器
	E-mail Alert
	RSS
	作者相关文章

关键词 ：概念识别, 深度迁移学习, 多标签分类, 医学图像检索, 主题模型

Abstract：Identifying useful concepts from large scale medical images is an important technology for image knowledge representation. Developing semantic concept detection algorithms is helpful to machine understanding and learning latent knowledge from medical images, and plays an important role in image-assisted diagnosis and intelligent image reading. In this study, the problem of detecting high-frequency concepts from medical images was transformed into a multi-label classification task. The deep transfer learning method based on convolutional neural network (CNNs) was used to recognize high-frequency medical concepts. The image retrieval-based topic modeling method was used to obtain the semantically related concepts from the similar images of given medical images. The CLEF cross language image retrieval track (ImageCLEF) launched the ImageCLEFcaption 2018 evaluation task on May, 2018, in which the Concept Detection subtask identified 111,156 semantic concepts from 222,314 training images and 9,938 test images. Our research group presented experimental results of both methods. The CNNs-based deep transfer learning method achieved the F1 score of 0.0928, which ranked second in all the submission teams. The retrieval-based topic model could recall some low-frequency concepts and achieved the F1 score of 0.0907, but dependent heavily on the image retrieval results. The results proved that the CNNs-based deep transfer learning method showed preferable robustness on high-frequency concept detection, but still a lot of room for improvement in the research of large-scale open semantic concept detection.

Key words： concept detection deep transfer learning multi-label classification medical image retrieval topic model

收稿日期: 2018-11-22

PACS:

R318

基金资助:中国医学科学院中央级公益性科研院所基本科研业务费项目(2017PT63010,2018PT33024);北京协和医学院中央高校基本科研业务费项目(3332018153)

通讯作者: E-mail: li.jiao@imicams.ac.cn

作者简介: #中国生物医学工程学会会员(Member, Chinese Society of Biomedical Engineering)

引用本文:

王序文, 张宇, 郭臻, 李姣. 医学图像语义概念识别方法研究[J]. 中国生物医学工程学报, 2019, 38(3): 306-314.
Wang Xuwen, Zhang Yu Guo, Zhen Li Jiao. Research on Methods of Semantic Concept Detection from Medical Images. Chinese Journal of Biomedical Engineering, 2019, 38(3): 306-314.

链接本文:

http://cjbme.csbme.org/CN/10.3969/j.issn.0258-8021.2019.03.007 或 http://cjbme.csbme.org/CN/Y2019/V38/I3/306

[1] 刘飞,张俊然,杨豪. 基于深度学习的医学图像识别研究进展[J]. 中国生物医学工程学报, 2018,37(1): 87-93.
[2] Litjens G, Kooi T, Bejnordi BE, et al. A survey on deep learning in medical image analysis [J]. Medical Image Analysis, 2017, 42(9): 60-88.
[3] He Kaiming, Zhang Xiangyu, Ren Shaoqing, et al. Deep residual learning for image recognition [C] // The IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778.
[4] Koniusz P, Yan Fei, Gosselin P H, et al. Higher-order occurrence pooling for bags-of-words: visual concept detection [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(2): 313-326.
[5] 李亚鹏,万遂人. 基于深度学习的行人属性多标签识别[J]. 中国生物医学工程学报, 2018, 37(4): 423-428.
[6] 孔小函,檀韬,包凌云,等. 基于卷积神经网络和多信息融合的三维乳腺超声分类方法[J]. 中国生物医学工程学报, 2018, 37(4): 414-422.
[7] 王翀,何兴鑫,方乐缘,等. 基于联合决策卷积神经网络的光学相干断层扫描图像自动分类[J]. 中国生物医学工程学报, 2018, 37(6): 641-648.
[8] Pan Jialin, Yang Qiang. A survey on transfer learning [J]. IEEE Transactions on Knowledge and Data Engineering, 2010, 22(10): 1345-1359.
[9] Esteva A, Kuprel B, Novoa RA, et al. Dermatologist-level classification of skin cancer with deep neural networks [J]. Nature, 2017, 542(7639): 115-118.
[10] 于玉海,林鸿飞,孟佳娜,等. 跨模态多标签生物医学图像分类建模识别[J]. 中国图像图形学报, 2018, 23(6): 917-927.
[11] ImageCLEF. ImageCLEFcaption 2017 Homepage. [EB/OL]. http://www.imageclef.org/2017/caption, 2017-09-14/2018-11-16.
[12] ImageCLEF. ImageCLEFcaption 2018 Homepage. [EB/OL]. http://www.imageclef.org/2018/caption, 2018-09-14/2018-11-16
[13] NCBI. PubMed. [EB/OL]. https://www.ncbi.nlm.nih.gov/pmc/, 2019-2-26/2019-03-22.
[14] U.S. National Library of Medicine. Unified Medical Language System (UMLS). [EB/OL]. https://www.nlm.nih.gov/research/umls/, 2018-10-11/2019-03-22.
[15] Bodenreider O. The unified medical language system (UMLS): integrating biomedical terminology [J]. Nucleic Acids Research, 2004,32:267-270.
[16] Eickhoff C, Schwall I, Seco de Herrera, et al. Overview of ImageCLEFcaption 2017 - image caption prediction and concept detection for biomedical images [C] //Lecture Notes in Computer Science: Experimental IR meets multilinguality, multimodality, and interaction. 8th International Conference of the CLEF Association (CLEF 2017). Dublin, Ireland. Cham: Springer, 2017: 315- 337.
[17] Seco de Herrera A.G, Eickhoff C, Andrearczyk V, et al. Overview of the image CLEF 2018 caption prediction tasks[C]// ImageCLEF - Multimedia Retrieval in CLEF. Avignon, France: CEUR Workshop, 2018: 2125.
[18] Pinho E, Figueira J, Silva JM, et al. Towards representation learning for biomedical concept detection in medical images: UA.PT bioinformatics in ImageCLEF 2017. [EB/OL]. http://120.52.51.13/ceur-ws.org/Vol-1866/paper_149.pdf. 2017-07-11/2019-03-22.
[19] Dimititris K, Ergina K. Concept detection on medical images using deep residual learning network. [EB/OL]. http://120.52.51.13/ceur-ws.org/Vol-1866/paper_122.pdf. 2017-07-13/2019-03-22.
[20] Valavanis L, Stathopoulos S. IPL at imageCLEF 2017 concept detection task. [EB/OL]. http://www.dei.unipd.it/～ferro/CLEF-WN-Drafts/CLEF2017/paper_144.pdf. 2017-07-11/2019-03-22.
[21] Rahman M, Lagree T, Taylor M. A cross-modal concept detection and caption prediction approach in ImageCLEFcaption track of ImageCLEF 2017. [EB/OL]. http://59.80.44.98/ceur-ws.org/Vol-1866/paper_160.pdf. 2017-07-11/2019-03-22.
[22] Ny Hoavy N, Mothe J, Randrianarivony MI. IRIT & MISA at image CLEF 2017-multi label classification. [EB/OL]. http://59.80.44.100/ceur-ws.org/Vol-1866/paper_148.pdf. 2017-07-11/2019-03-22.
[23] Soldaini L, Goharian N. QuickUMLS: A fast, unsupervised approach for medical concept extraction. [EB/OL]. http://medir2016.imag.fr/data/MEDIR_2016_paper_16.pdf. 2016-07-27/2019-03-22.
[24] Szegedy C, Vanhoucke V, Ioffe S, et al. Rethinking the inception architecture for computer vision[C]//The IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas: IEEE Computer Society, 2016:2818-2826.
[25] Russakovsky O, Deng Jia, Su Hao, et al. Imagenet large scale visual recognition challenge [J]. International Journal of Computer Vision, 2015, 115(3): 211-252.
[26] Mathias Lux. LIRE: Lucene Image Retrieval. [EB/OL]. http://www.lire-project.net/. 2015-8-21/2018-11-16
[27] Gan R, Yin J. Using LIRe to implement image retrieval system based on multi-feature descriptor[C]// The 3rd International Conference on Digital Manufacturing & Automation. Guilin: IEEE Computer Society, 2012: 1014-1017.
[28] Blei DM, Ng AY, Jordan MI. Latent dirichlet allocation [J]. Journal of Machine Learning Research, 2003, 3:993-1022.
[29] Pinho E, Costa C. Feature learning with adversarial networks for concept detection in medical images: UA.PT bioinformatics at ImageCLEF 2018[C]// ImageCLEF - Multimedia Retrieval in CLEF CEUR Workshop Proceedings. Avignon: CEUR-WS, 2018:10-14.