Abstract：2D/3D registration is widely used in clinical diagnosis and surgical navigation planning,which can solve the problem of missing information in different dimensions of medical images and assist doctors to accurately locate patients′ lesions during surgery. The conventional 2D/3D registration method mainly relies on the gray level of the image for registration,but the registration process is very time consuming,which is not conducive to the clinical real-time requirements,and the registration process is easy to fall into the local optimum. This study proposed a deep learning approach to solve 2D/3D medical image registration problems. The method used a deep learning-based convolutional neural network to train the DRR and automatically learned image features to predict the parameters corresponding to the X-ray image to achieve registration. In the study,the human pelvis model bone was used as the experimental object. A total of 36,000 DRR images were generated as training sets,and 50 X-ray images of the model bone were collected by C arm for verification. Results showed that the test values for the three precision evaluation indicators of the correlation coefficient,normalized mutual information and Euclidean distance were 0.82±0.07,0.32±0.03,61.56±10.91 and the corresponding test values of the conventional 2D/3D algorithm were 0.79±0.07,0.29±0.03,37.92±7.24. These results meant the registration accuracy of deep learning algorithm was better than the conventional 2D/3D algorithm and there was no local optimal value for deep learning algorithm. Meanwhile,the registration time of deep learning was about 0.03 s,which was much lower than the time of conventional 2D/3D registration,which can satisfy the clinical demand for real-time registration. In the future,2D/3D deep learning registration research of clinical data will be further carried out.
[1] Markelj P,Tomazevic D,Likar B,et al.A review of 3D/2D registration methods for image-guided interventions[J].Medical Image Analysis,2012,16(3):642-661. [2] 王雷,高欣,崔学理,等.基于灰度距离融合的2D/3D刚性配准[J].光学精密工程,2014,22(10):2815-2824. [3] Jannin P,Fitzpatrick JM,Hawkes D,et al.Validation of medical image processing in image-guided therapy[J].IEEE Transactions on Medical Imaging,2002,21(12):1445-1449. [4] Hashizume M,Yasunaga T,Tanoue K,et al.New real-time MR image-guided surgical robotic system for minimally invasive precision surgery[J].International Journal of Computer Assisted Radiology and Surgery,2008,2(6):317-325. [5] Jaffray D,Kupelian P,Djemil T,et al.Review of image-guided radiation therapy[J].Expert Review of Anticancer Therapy,2007,7(1):89-103. [6] Van d KEB,Penney GP,Tomazevic D,et al.Standardized evaluation methodology for 2-D-3-D registration[J].IEEE Transactions on Medical Imaging,2005,24(9):1177-1189. [7] Markelj P,Tomazevic D,Pernus F,et al.Robust gradient-based 3-D/2-D registration of CT and MR to X-ray images[J].IEEE Transactions on Medical Imaging,2009,27(12):1704-1714. [8] Myronenko A,Song X.Point set registration:Coherent point drift[J].IEEE Transactions on Pattern Analysis &Machine Intelligence,2010,32(12):2262-2275. [9] Birkfellner W,Stock M,Figl M,et al.Stochastic rank correlation:A robust merit function for 2D/3D registration of image data obtained at different energies[J].Medical Physics,2009,36(8):3420-3428. [10] Gendrin C,Markelj P,Pawiro SA,et al.Validation for 2D/3D registration II:The comparison of intensity and gradient based merit functions using a new gold standard data set[J].Medical Physics,2011,38(3):1491-1502. [11] Russakoff DB,Rohlfing T,Adler JR,et al.Intensity-based 2D-3D spine image registration incorporating a single fiducial marker1[J].Academic Radiology,2005,12(1):37-50. [12] 党建武,杭利华,王阳萍,等.基于GPU的2D-3D医学图像配准[J].计算机科学,2013,40(4):306-309. [13] Li X,Yang J,Zhu Y.Digitally reconstructed radiograph generation by an adaptive Monte Carlo method[J].Physics in Medicine &Biology,2006,51(11):2745. [14] 梁玮.2D-3D医学图像配准研究[D].南京:东南大学,2004. [15] Spoerk J,Bergmann H,Wanschitz F,et al.Fast DRR splat rendering using common consumer graphics hardware[J].Medical Physics,2007,34(11):4302-4308. [16] 付东山.一种二维-三维医学图像配准方法及系统[P]:中国专利:102222330.2013-09-25. [17] 顾吉.基于图形卡的二维/三维配准的算法实现[D].上海:上海交通大学,2007. [18] Munbodh R,Tagare HD,Chen Z,et al.2D-3D registration for prostate radiation therapy based on a statistical model of transmission images[J].Medical Physics,2009,36(10):4555-4568. [19] Dey J,Napel S.Targeted 2D/3D registration using ray normalization and a hybrid optimizer[J].Medical Physics,2006,33(12):4730-4738. [20] Knaan D,Joskowicz L.Effective intensity-based 2D/3D rigid registration between fluoroscopic X-ray and CT[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention.Berlin:Springer,2003:351-358. [21] Turgeon GA,Lehmann G,Guiraudon G,et al.2D-3D registration of coronary angiograms for cardiac procedure planning and guidance[J].Medical Physics,2005,32(12):3737-3749. [22] Szegedy C,Liu W,Jia Y,et al.Going deeper with convolutions[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Boston:IEEE,2015:1-9. [23] Szegedy C,Vanhoucke V,Ioffe S,et al.Rethinking the Inception Architecture for Computer Vision[C]//IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.:IEEE Computer Society,2016:2818-2826. [24] Wen Y,Zhang K,Li Z,et al.A Discriminative feature learning approach for deep face recognition[C]//Computer Vision-ECCV 2016.Berlin:Springer International Publishing,2016:499-515. [25] 陈畅,程少杰,李卫滨,等.基于卷积神经网络的外周血白细胞分类[J].中国生物医学工程学报,2018,37(1):17-24. [26] Rohlfing T,Russakoff DB,Denzler J,et al.Progressive attenuation fields:Fast 2D-3D image registration without precomputation[J].Medical Physics,2005,32(9):2870. [27] Otake Y,Armand M,Armiger RS,et al.Intraoperative image-based multiview 2D/3D registration for image-guided orthopaedic surgery:Incorporation of fiducial-based C-arm tracking and GPU-acceleration[J].IEEE Transactions on Medical Imaging,2012,31(4):948-962. [28] Han X.Feature-constrained nonlinear registration of lung CT images[C]//Medical Image Analysis for The Clinic:A Grand Challenge.Heidelberg:Springer Nature,2010:63-72. [29] 张翼,王满宁,宋志坚.脊柱手术导航中分步式2D/3D图像配准方法[J].计算机辅助设计与图形学学报,2007,19(9):1154-1158.
