Image-Guided Arthroscopic Surgery Based on Virtual Endoscopic Technology
Cui Xiwen1, Chen Fang2, Han Boxuan1, Ma Cong1, Ma Longfei1, Liao Hongen1#*
1(Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing 100084, China) 2(Department of Computer Science and Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing 211106, China)
Abstract：Knee arthroscopic surgery is a kind of minimally invasive surgery using endoscope, which is associated with small incisions and a shorter hospital stay. While in the meantime, arthroscopic surgery uses two dimensional image as visual guidance, thus brings the drawback of lacking depth information, getting blocked easily, counting on surgeon′s experience. Our research brought up an arthroscopic surgery guiding system based on virtual-vision technology with pre-operation imaging. Through this system, pre-operation image collaborated with intraoperative image, and thus made the surgery procedure more convenient and reliable. Virtual-vision rendering method was used for arthroscopic surgery guidance. The calibration procedure for the arthroscope and the registration procedure for the whole system were finished at first. Then the target of virtual rendering for the pre-operation imaging in corresponding viewing position was achieved, during which, real-time tracking for the arthroscope was essential. We set up the arthroscopic surgery guiding system based on external tracking information and pre-operation imaging. Software interface for the system was developed with computer vision rendering algorithm. A knee model experiment was conducted to validate the system, where the mean square error (MSE) for the calibration of tracking device was within 1 mm,and the MSE for the image fusion was below 0.7 mm, both the separate and fusion image for guidance was provided in real-time. The surgery guiding system described in this research could provide enhanced image guidance for knee arthroscopic surgery. Multi-source imaging was made good use in our system, which would provide the surgeons with direct and precise surgery guiding information.
崔曦雯, 陈芳, 韩博轩, 马聪, 马龙飞, 廖洪恩. 虚拟内窥镜图像增强膝关节镜手术导航系统[J]. 中国生物医学工程学报, 2019, 38(5): 558-565.
Cui Xiwen, Chen Fang, Han Boxuan, Ma Cong, Ma Longfei, Liao Hongen. Image-Guided Arthroscopic Surgery Based on Virtual Endoscopic Technology. Chinese Journal of Biomedical Engineering, 2019, 38(5): 558-565.
[1] 王凤娟, 陈翠玲, 林穗红. 膝关节镜手术配合 [J]. 中华生物医学工程杂志, 2004, 10(4):349-349. [2] 敖英芳, 王健全, 余家阔, 等. 膝关节镜下前交叉韧带重建术 [J]. 中国运动医学杂志, 2000, 19(3):231-231. [3] Peters TM. Image-guidance for surgical procedures [J]. Physics in Medicine and Biology, 2006,51(14): R505-R540. [4] Marescaux J, Diana M, Next step in minimally invasive surgery: hybrid image-guided surgery [J]. Journal of Pediatric Surgery, 2015, 50(1):30-36. [5] 裴大婷, 黄德群, 陈军,等. 手术导航系统的研究现状与发展趋势 [J]. 临床医学工程, 2017, 24(9):1326-1328. [6] Namba H, Tokuyama T, Yokota N, et al. Image-Guided surgery for gliomas [M]//Minimally Invasive Neurosurgery and Multidisciplinary Neurotraumatology. Tokyo:Springer, 2006:22-28. [7] Evans L, Olson D, Cai Yunliang, et al. Stereovision co-registration in image-guided spinal surgery: accuracy assessment using explanted porcine spines [J]. Operative Neurosurgery, 2018,15(6):686-691 [8] Citardi MJ, Agbetoba A, Bigcas JL, et al. Augmented reality for endoscopic sinus surgery with surgical navigation: a cadaver study [J]. International Forum of Allergy & Rhinology, 2016, 6(5):523-528. [9] Fried M. Image-guided endoscopic surgery: results of accuracy and performance in a multicenter clinical study using an electromagnetic tracking system [J]. Laryngoscope, 1997, 107:594-601. [10] Liao Hongen, Tsuzuki M, Mochizuki T, et al. Fast image mapping of endoscopic image mosaics with three-dimensional ultrasound image for intrauterine fetal surgery [J]. Minimally Invasive Therapy & Allied Technologies, 2009, 18(6):332-340. [11] Ando T, Kawashima D, Kim H, et al. Direct minimally invasive intraoperative electrophysiological mapping of the heart [J]. Minimally Invasive Therapy & Allied Technologies, 2013, 22(6):372-380. [12] Wang Junchen, Suenaga H, Yang Liangjing, et al. Video see-through augmented reality for oral and maxillofacial surgery [J]. The International Journal of Medical Robotics and Computer Assisted Surgery, 2016, 13(2): e1754. [13] Luo Xiaobao, Mori K, Peters TM. Advanced endoscopic navigation: surgical big data, methodology, and applications [J]. Annual Review of Biomedical Engineering, 2018, 20(1):221-251 [14] Mori K, Urano A, Hasegawa JI, et al. Virtualized endoscope system-An application of virtual reality technology to diagnostic aid [J]. IEICE Trans on Information and Systems, 1996, E79d(6): 809-819. [15] Mourgues F, Frédéric D, >Ève C. 3D reconstruction of the operating field for image overlay in 3D-endoscopic surgery [C]// IEEE & Acm International Symposium on Augmented Reality. New York: IEEE Computer Society, 2001:191-192. [16] Zhang Zhengyou. A flexible new technique for camera calibration [J]. IEEE Trans on Pattern Analysis and Machine Intelligence, 2000, 22(11):1330-1334. [17] Devernay F, Faugeras OD, Automatic calibration and removal of distortion from scenes of structured environments [J]. Proc SPIE, 1995, 2567:62-72. [18] Weng Juyang, Cohen P, Herniou M. Camera calibration with distortion models and accuracy evaluation [J]. IEEE Trans on Pattern Analysis and Machine Intelligence, 1992, 14(10): 965-980 [19] Tsai RY, Lenz RK. A new technique for fully autonomous and efficient 3D robotics hand/eye calibration [J]. IEEE Trans on Robotics and Automation, 1989, 5(3):345-358. [20] Liao Hongen, Inomata T, Sakuma I, et al. 3-D augmented reality for MRI-guided surgery using integral videography autostereoscopic image overlay [J]. IEEE Trans on Biomedical Engineering, 2010, 56(6): 1476-1486. [21] 徐峰, 刘伟, 李传富, 等. 具有大形变特征的颅脑CT图像的非刚性配准 [J]. 中国生物医学工程学报, 2010, 29(2):172-177. [22] 张召瑞, 张旭, 郑泽龙, 等 融合旋转平移信息的机器人手眼标定方法 [J]. 仪器仪表学报, 2015, 36(11):2443-2450. [23] Ishitani K, Daisuke D, Kitasaka T, et al. Easy and stable bronchoscope camera calibration technique for bronchoscope navigation system [C]// Proc SPIE. San Diego: SPIE, 2007, 6509:1-12. [24] Usenko V, Engel J, Stuckler J, et al. Direct visual-inertial odometry with stereo cameras[C]// 2016 IEEE International Conference on Robotics and Automation (ICRA). Stockholm: IEEE, 2016:1885-1892. [25] 张少敏, 支力佳, 赵大哲, 等. 融合SIFT特征的熵图估计医学图像非刚性配准 [J]. 中国图象图形学报, 2018, 17(3):115-121. [26] Marmulla R, Hassfeld S, Lüth T, et al. Soft tissue scanning for patient registration in image-guided surgery [J]. Computer Aided Surgery, 2003, 8(2):70-81.
