Fusion of Multi-Modality Medical Imaging Information for Clinical Decision in Tumor Diagnosis and Treatment
1 Sino-Dutch Biomedical and Information Engineering School, Northeastern University, Shenyang 110819, China
2 Department of Radiology, Shengjing Hospital of China Medical University, Shenyang 110004, China
Abstract：Malignant tumors are threatening human health seriously. Early diagnosis and effective treatment can reduce the mortality rate. Taking advantage of in-vivo and non-invasive measurements, modern biomedical imaging is able to provide tumor diagnosis and treatment with massive valuable information on both structure and function. However, cancer is a multi-factorial and development disorder, meanwhile each modality of imaging owns its drawback and limitations. Therefore, it is hardly to elucidate the tumor′s intrinsic mechanism and developing process via a single-modality image acquisition. Multi-modal medical image fusion, which can realize the information complement and cross-validation, is considered to be the promising choice. In this review, after analyzing the characteristics of each imaging technique, the stateoftheart of registration, image fusion methods and their clinic applications are reviewed, and then the available problems and potential future direction are discussed.
齐守良1岳勇2辛军2康雁1*. 面向临床肿瘤诊疗决策的多模态医学影像融合[J]. 中国生物医学工程学报, 2013, 32(3): 356-362.
QI Shou Liang1 YUE Yong2 XIN Jun2 KANG Yan1*. Fusion of Multi-Modality Medical Imaging Information for Clinical Decision in Tumor Diagnosis and Treatment. journal1, 2013, 32(3): 356-362.
［1］Ferlay J, Shin HR, Bray F, et al. Globocan 2008: Cancer incidence and mortality worldwide ［EB/OL］. http://globocan.iarc.fr/ 2011-12-14/2013-04-10.
［2］Siegel R, Naishadham D, Jemal A. Cancer statistics 2012 ［J］. CA: A Cancer Journal for Clinicians, 2012, 62(1): 10-29.
［3］Berry DA, Cronin KA, Plevritis SK, et al. Effect of screening and adjuvant therapy on mortality from breast cancer ［J］, The New England Journal of Medicine, 2005, 353(17): 1784-1792.
［4］Edwards BK, Ward E, Kohler BA, et al. Annual report to the nation on the status of cancer, 1975-2006, featuring colorectal cancer trends and impact of interventions (risk factors, screening, and treatment) to reduce future rates ［J］. Cancer, 2010, 116(3): 544-573.
［5］Etzioni R, Tsodikov A, Mariotto A, et al. Quantifying the role of PSA screening in the US prostate cancer mortality decline ［J］. Cancer Causes Control, 2008, 19(2): 175-181.［6］汤钊猷，现代肿瘤学 ［M］.（第3版）. 上海: 复旦大学出版社, 2011: 1-11.
［7］赵平，蔡坚强,主译，肝胆肿瘤 ［M］. 北京: 中国医药科技出版社，2010: 1-75.
［8］Willyard C. Imaging: seeing is believing ［J］. Nature, 2011, 480(7377): 52-53.
［9］Ehman RL, Hendee WR, Welch MJ. Blueprint for imaging in biomedical research ［J］, Radiology, 2007, 244(1): 12-27.
［10］Fass L. Imaging and cancer: a review ［J］. Molecular Oncology, 2008, 2(2): 115-152.
［11］Asselin MC, O’Connor JPB, Boellaard R, et al. Quantifying heterogeneity in human tumors using MRI and PET ［J］. European Journal of Cancer, 2012, 48(4): 447-455.
［12］Waldman AD, Jackson A, Price SJ, et al. Quantitative imaging biomarkers in neurooncology ［J］. Nature Review Clinical Oncology, 2009, 6(8): 445-454.
［13］Pannu HK, Jacobs JE, Lai S, et al. Coronary CT angiography with 64 MDCT: assessment of vessel visibility ［J］. AJR American Journal of Roentgenology, 2006, 187(1): 119-126.
［14］Lee MW, Lee JM, Lee JY, et al. Preoperative evaluation of the hepatic vascular anatomy in living liver donors: comparison of CT angiography and MR angiography ［J］. Journal of Magnetic Resonance Imaging, 2006, 24(5): 1081-1087.
［15］Hoh CK. Clinical use of FDG PET ［J.］ Nuclear Medicine and Biology, 2007, 34(7): 737-742.
［16］Marcus CD, Marcus VL, Cucu C, et al. Imaging techniques to evaluate the response to treatment in oncology: current standards and perspectives ［J］. Critical Reviews in Oncology/Hematology, 2009, 72(3): 217-238.
［17］Bihan DL, Mangin JF, Poupon C, et al. Diffusion tensor imaging: concepts and applications ［J］. Journal of Magnetic Resonance Imaging, 2001, 13(4):534-546.
［18］Koh DM, Collins DJ. Diffusion-weighted MRI in the body: applications and challenges in oncology ［J］. America Journal of Radiology, 2007, 188(6): 1622-1635.
［19］Patterson DM, Padhani AR, Collins D. Technology insight: water diffusion MRIa potential new biomarker of response of cancer therapy ［J］. Nature Clinical Practice Oncology, 2008, 5(4): 220-233.
［20］Chandarana H, Taouli B. Diffusion and perfusion imaging of the liver ［J］. European Journal of Radiology, 2010, 76(3): 348-358.
［21］Mariappan YK, Glaser KJ, Ehman RL. Magnetic resonance elastography: a review ［J］. Clinical anatomy, 2010, 23(5): 497-511.
［22］Ieva AD, Grizzi F, Rognone E. Magnetic resonance elastography: a general overview of its current and future applications in brain imaging ［J］. Neurosurgery Review, 2010, 33(2): 137-145.
［23］Hines CDG, Bley TA, Lindstrom MJ. Repeatability of magnetic resonance elastography for quantification of hepatic stiffness ［J］. Journal of Magnetic Resonance Imaging, 2010, 31(3): 725-731.
［24］Li YY, Wang XM, Zhang YX, et al. Ultrasonic elastography in clinical quantitative assessment of fatty liver ［J］. World Journal of Gastroenterolgy, 2010, 16(37): 4733-4737.［25］Hanahan D, Weinberg RA. Hallmarks of cancer: the next generation ［J］. Cell, 2011, 144(3): 646-674.
［26］www.siemens.com/healthcare
［27］陈武凡，秦安，姜少峰，等. 医学图像分析的现状与展望 ［J］. 中国生物医学工程学报, 2008, 27(2): 175-181.
［28］何友，信息融合理论及应用［M］. 北京: 电子工业出版社，2010: 1-19.
［29］韩崇昭，朱洪艳，段战胜，等. 多源信息融合［M］. 北京: 清华大学出版社，2010: 1-14.
［30］田捷，包尚联，周明全. 医学影像处理与分析 ［M］. 北京: 电子工业出版社，2003: 96-114.
［31］康晓东. 医学影像图像处理［M］. 北京: 人民卫生出版社，2009: 169-195.
［32］Pluim JPW, Maintz JBA, Viergever MA. Mutualinformationbased registration of medical images: a survey ［J］. IEEE Transaction on Medical Imaging, 2003, 22(8): 986-1004.
［33］Zitova B, Flusser J. Image registration methods: a survey ［J］. Image and Vision Computing, 2003, 21(11): 997-1000.
［34］Slomak PJ, Baum RP. Multimodality image registration with software: stateoftheart ［J］. European Journal of Nuclear Medicine Imaging, 2009, 36(S1): 44-55.
［35］Mattes D, Haynor DR, Vesselle H, et al. PETCT image registration in the chest using freeform deformations ［J］. IEEE Transaction on Medical Imaging, 2003, 22(1): 120-128.
［36］Holden M. A review of geometric transformations for nonrigid body registration ［J］. IEEE Transaction on Medical Imaging, 2008, 27(1): 111-127.
［37］Rueckert D, Aljabar P. Nonrigid registration of medical images: theory, methods, and applications ［J］. IEEE Signal Processing Magazine, 2010, 27(4): 113-119.
［38］李雄飞，张存利，李鸿鹏. 医学图像配准技术进展 ［J］. 计算机科学，2010，37(7): 27-33.
［39］张少敏, 支力佳, 赵大哲. 融合梯度信息的最小生成树医学图像配准［J］. 东北大学学报(自然科学版), 2010, 31(10): 1393-1396.
［40］王安娜，薛嗣麟，俞跃. 基于改进光流场模型的医学图像配准方法［J］.中国图象图形学报，2010，15(2): 328-333.
［41］李文龙，程流泉，李军. 基于自由形变的3D非线性医学图像配准［J］. 中国医学影像技术，2011，27(12): 2536-2540.
［42］党建武，孙滕，王阳萍. 互信息驱动的有限元医学图像配准方法［J］. 计算机应用，2011，31(3): 733-736.
［43］许鸿奎，江铭炎，杨明强. 基于多目标交叉变异粒子群算法的多模医学图像非刚性配准［J］. 中国生物医学工程学报，2011，30(2): 232-239.
［44］Zhang XH, Arridge S, Hawkes DJ, et al. A nonrigid registration framework using spatially encoded mutual information and freeform deformations ［J］. IEEE Transaction on Medical Imaging, 2011, 30(10): 1819-1828.
［45］Khader M, Hamza AB. Nonrigid Image registration using an entropic similarity ［J］. IEEE Transaction on Information Technology in Biomedicine, 2011, 15(5): 681-690.
［46］Chen HZ, Kingsbury N. Efficient registration of nonrigid 3D bodies ［J］. IEEE Transaction on Image Processing, 2012, 21(1): 262-272.
［47］敬忠良，肖刚，李振华. 图像融合理论与应用 ［M］. 北京: 高等教育出版社，2007: 47-96.
［48］那彦，焦李成，基于多分辨率分析理论的图像融合方法 ［M］. 西安: 西安电子科技大学出版社，2007: 83-93.
［49］Pan XG, Wang MY, Liu JR, et al. An automatic registration approach for whole body PET images and head MR images ［C］// Chuo KC., eds. The 5th International Conference on Bioinformatics and Biomedical Engineering, New York: Curran Associates, 2011: 1-4.
［50］Pajares G, Cruz JM. A waveletbased image fusion tutorial ［J］. Pattern Recognition, 2004, 37: 1855-1872.
［51］Goshtasby AA, Nikolov S. Image fusion: advances in the state of the art ［J］. Information Fusion, 2007, 8: 114-118.
［52］陈爽，姜威. 基于DT2DWT 的医学图像融合算法［J］. 山东大学学报(工学版)，2008，38(1): 33-35.
［53］胡俊峰，唐鹤云，钱建生. 基于小波变换医学图像融合算法的对比分析 ［J］. 中国生物医学工程学报，2011，30(2): 196-205.
［54］王安娜，吴洁，张新华. 一种基于正交小波包分解和2vSVM 的医学图像融合新方法［J］. 小型微型计算机系统，2007，28(12): 2268-2271.
［55］徐显，杨杰. 基于图形处理器硬件加速的高精度医学图像融合算法［J］. 上海交通大学学报，2010，44(2): 248-251.
［56］Piella G. A general framework for multiresolution image fusion: from pixels to regions ［J］. Information Fusion, 2003, 4(4): 259-280.
［57］Lewis JJ, Callaghan RJO, Nikolov SG., et al. Pixel and regionbased image fusion with complex wavelets ［J］. Information Fusion, 2007, 8(2): 119-130.
［58］Shapiro LG, Haralick RM. Organization of relational models for scene analysis ［J］. IEEE Transaction on Pattern Analysis Machine Intelligence, 1982, 4(6): 595-602.
［59］Williams ML, Wilson RC, Hancock ER. Deterministic search for relational graph matching ［J］. Pattern Recognition, 1999, 32(7): 1255-1516.
［60］Beyer T, Townsend DW, Brun T, et al. A combined PET/CT scanner for clinical oncology ［J］. Journal of Nuclear Medicine, 2001, 41(8): 1369-1379.
［61］Townsend DW, Carney JPJ, Yap JT, et al. PET/CT today and tomorrow ［J］. Journal of Nuclear Medicine, 2004, 45(S1): 4-14.
［62］Weber WA, Crosu AL, Czernin J. Technology insight: advances in molecular imaging and an appraisal of PET/CT scanning ［J］. Nature Clinical Practice Oncology, 2007, 5(3): 220-233.
［63］Czernin J, Auerbach MA, Schelbert HR. Improvement in cancer staging with PET/CT: literaturebased evidence as of September 2006 ［J］. Journal of Nuclear Medicine, 2007, 48(S1): 78-88.
［64］Antoch G, Bockisch A. Combined PET/MRI: a new dimension in wholebody oncology imaging? ［J］. European Journal of Nuclear Medicine Molecular Imaging, 2009, 36(S1): 113-120.
［65］Raabe A. Laser surface scanning for patient registration in intracranial imageguided surgery ［J］. Neurosurgery, 2002, 54(4): 797-803.
［66］Audette MA, Siddiqi K, Ferrie FP, et al. An integrated rangesensing, segmentation and registration framework for the characterization of intrasurgical brain deformation in imageguided surgery ［J］. Computer Vision Image Understanding, 2003, 89(2-3): 226-251.
［67］Huq S, Mayles P, Besa C, et al. Transition from 2D radiotherapy to 3D conformal and intensitymodulated radiotherapy, IAEATECDOC1588, International Atomic Energy Agency, 2008.
［68］Brunt JN. Computer Tomography Magnetic Resonance Image registration in radiotherapy treatment planning ［J］. Clinical Oncology, 2010, 22(8): 688-697.
［69］Demongeot J, Wendling JB, Mattes J, et al. Multiscale modeling and imaging: the challenges of biocomplexity ［J］. Proceedings of the IEEE, 2003, 91(10): 1723-1737.
［70］史忠植. 知识发现［M］. 北京: 清华大学出版社，2011 1-19.
［71］曹建文. 临床决策分析哈佛版［M］.上海: 复旦大学出版社，2005: 1-10.
