Survey of SeeThrough Technique in ComputerAided Surgery
1 School of Control Science and Engineering, Shandong University, Jinan 250061, China
2 School of Computer Science, Bangor University, Bangor, LL57 1UT, UK
Abstract:The application of seethrough technique in computeraided surgery is a hot research topic at present. In this article, firstly, the concept of seethrough is introduced, and the necessity of its application in computeraided surgery is also emphasized. Secondly, the state of arts of seethrough technique is discussed systematically in computeraided surgery. Thirdly, the research contents of seethrough technique are summarized into three parts, including registration of medical imaging data, parameterestimation of seethrough model, and depth perception, which are all explained in details. Fourthly, two standard systems based on seethrough technique are addressed in this paper. Finally, through the analysis on technique, human factor and clinic applications, further research questions and possible directions in the future are proposed.
[1]Endsley MR. Toward a theory of situation awareness in dynamic system [J]. Human Factors, 1995, 37(1): 32-64.
[2]State A, Ackerman J, Hirota G, et al. Dynamic virtual convergence for video seethrough headmounted displays: maintaining maximum stereo overlap throughout a closerange work space [C] // Proceedings of 4th International Symposium on Augmented Reality. Washington: IEEE Computer Society, 2001: 137-146.
[3]State A, Keller KP, Fuchs H. Simulationbased design and rapid prototyping of a parallaxfree orthoscopic video seethrough headmounted display [C]// Proceedings of 4th IEEE and ACM International Symposium on Mixed and Augmented Reality. Vienne: IEEE Computer Society, 2005: 28-31.
[4]Sauer F, Khamene A, Bascle B, et al. A headmounted display system for augmented reality image guidance: towards clinical evaluation for iMRIguided neurosurgery [C]// Niessen WJ, Viergever MA, eds. Proceedings of 4th International Conference on Medical Image Computing and Computer Assisted Intervention. Utrecht: SpringerVerlag, 2001: 707-716.
[5]Maurer CR, Sauer F, Hu Bo, et al. Augmentedreality visualization of brain structures with stereo and kinetic depth cues: system description and initial evaluation with head phantom [C]// Seong KM, eds. Proceedings of SPIE 4319, Medical Imaging 2001: Visualization, Display, and ImageGuided Procedures. San Diego: SPIE, 2001: 445-456.
[6]Traub J, Stefan P, Heining SM, et al. Hybrid navigation interface for orthopedic and trauma surgery [C]// Larsen R, Nielsen M, Sporring J, eds. Proceedings of 9th International Conference on Medical Image Computing and Computer Assisted Intervention. Copenhagen: SpringerVerlag, 2006: 373-380.
[7]Rhienmora P, Gajananan K, Haddawy P, et al. Augmented reality haptics system for dental surgical skills training [C]// Baciu G, Lau RWH, Lin Ming, et al, eds. Proceedings of 17th ACM Symposium on Virtual Reality Software and Technology. Hong Kong: ACM, 2010: 97-98.
[8]Birkfellner W, Figl M, Huber K, et al. A headmounted operating binocular for augmented reality visualization in medicinedesign and initial evaluation [J]. IEEE Trans Medical Imaging, 2002, 21(8): 991-997.
[9]Figl M, Ede C, Hummel J, et al. A fully automated calibration method for an optical seethrough headmounted operating microscope with variable zoom and focus [J]. IEEE Trans Medical Imaging, 2005, 24(11): 1492-1499.
[10]Birkfellner W, Figl M, Huber K, et al. The varioscope AR—A headmounted operating microscope for augmented reality [C]// Delp SL, DiGioia AM, Jaramaz B, eds. Proceedings of 3th International Conference on Medical Image Computing and Computer Assisted Intervention. Pittsburgh: SpringerVerlag, 2000: 869-877.
[11]Lin Yenkun, Tsai Kunlun, Yau Hongtzong. The development of optical seethrough display based on augmented reality for oral implant surgery simulation [J]. ComputerAided Design and Applications, 2012, 9(1): 111-120.
[12]Edwards PJ, King AP, Maurer CR, et al. Design and evaluation of a system for microscopeassisted guided interventions (MAGI) [J]. IEEE Trans Med Imaging, 2000, 19(11):1082-1093.
[13]Shahidi R, Bax MR, Maurer CR, et al. Implementation, calibration and accuracy testing of an imageenhanced endoscopy system [J]. IEEE Trans Med Imaging, 2002, 21(12):1082-1093.
[14]Mourgues F, Viebille T, Falk V, CosteManiere E. Interactive guidance by image overlay in robot assisted coronary artery bypass [C]// Ellis RE, Peters TM, eds. Proceedings of 6th International Conference on Medical Image Computing and Computer Assisted Intervention. Montreal: SpringerVerlag, 2003: 173-181.
[15]Liao Hongen, Iwahara M, Koike T, et al. Scalable highresolution integral videograohy autostereoscopic display with a seamless multiprojection system [J]. Applied Optics, 2005, 44(3): 305-315.
[16]Liao Hongen, Hata H, Nakajima S, et al. Surgical navigation by autostereoscopic image overlay of integral videography [J]. IEEE Trans Information Technology in Biomedicine, 2004, 8(2): 114-121.
[17]Liao Hongen, Inomata T, Sakuma I, et al. Surgical navigation of integral videography image overlay for open MRIguided glioma surgery [C]// Yang Guangzhong, eds. Proceedings of 3rd international workshop on Medical Imaging and Augmented Reality. Shanghai: SpringerVerlag, 2006: 187-194.
[18]Edward H. Livingston. Surgical and clinical needs for DLP hyperspectral imaging [C]// Douglass MR, Hornback LJ, eds. Proceedings of SPIE Emerging Digital Micromirror Device Based Systems and Applications II. San Francisco: SPIE, 2010: 1-9.
[19]Masamune K, Fichtinger G, Deguet A, et al. An image overlay system with enhanced reality for percutaneous therapy performed inside CT scanner [C]// Dohi T, Kikinis R, eds. Proceedings of 5th International Conference on Medical Image Computing and Computer Assisted Intervention. Tokyo: SpringerVerlag, 2002: 77-84.
[20]Fichtinger G, Deguet A, Masamune K, et al. Image overlay guidance for needle insertion in CT scanner [J]. IEEE Trans Biomedical Engineering, 2005, 52(8): 1415-1424.[21]Weiss CR, Marker DR, Fischer GS, et al. Augmented reality visualization using imageoverlay for MRguided intervention: system description, feasibility, and initial evaluation in a spine phantom [J]. American Journal of Roentgenology, 2011, 196(1): 305-307.
[22]Tagaya N, Aoyagi H, Nakagawa A, et al. A novel approach for sentinel lymph node identification using fluorescence imaging and image overlay navigation surgery in patients with breast caner [J]. World Journal of Surgery, 2011, 35(2): 154-158.
[23]Tuceryan M, Genc Y, Navab N. Single point active alignment method (SPAAM) for optical seethrough HMD calibration for AR [J]. Presence: Teleoperators and Virtual Environments, 2002, 11(3): 259-276.
[24]Axholt M, Cooper MD, Skoglund MA, et al. Parameter estimation variance of the single point active alignment method in optical seethrough head mounted display calibration [C]// Hirose M, Lok B, Majumder A, Schmalstieg D, eds. Proceedings of IEEE Virtual Reality. Singapore: IEEE Computer Society, 2011: 27-34.
[25]Gilson S, Fitzgibbon AW, Glennerster A. Spatial calibration of an optical seethrough headmounted display [J]. Journal of Neuroscience Methods, 2008, 173(3): 140-146.
[26]Braux-Zin J, Bartoli A, Dupont R, et al. Calibrating an optical seethrough rig with two nonoverlapping cameras: The virtual camera framework [C]// Proceedings of 2nd Joint 3DIM/3DPVT Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission. Zurich: IEEE computer society, 2012: 308 -315.
[27]Prytz E, Montano M, Scerbo MW. Using Fitts' law for a 3D pointing task on a 2D display: Effects of depth and vantage point [C]// Proceedings of the Human Factors and Ergonomics Society. Boston: Sage Publications, 2012: 1391-1395.
[28]Jones JA, Swan II JE, Singh G, et al. Peripheral visual information and its effect on distance judgments in virtual and augmented environments [C]// McDonnell R, Thorpe SJ, Spencer SN, eds. Proceedings of ACM Symposium on Applied Perception in Graphics & Visualization. Toulouse: ACM, 2011: 29-36.
[29]Kuhl SA, Thompson WB, CreemRegehr SH. HMD calibration and its effects on distance judgments [J]. ACM Transactions on Applied Perception, 2009, 6(3): 1-20.
[30]Singh G, Swan II JE, Jones JA, et al. Depth judgments by reaching and matching in nearfield augmented reality [C]// Proceedings of IEEE Virtual Reality. Orange County: IEEE Computer Society, 2012: 165-166.
[31]Jones JA, Suma EA, Krum DM, et al. Comparability of narrow and wide fieldofview headmounted displays for mediumfield distance judgments [C]// Proceedings of ACM Symposium on Applied Perception. Los Angeles: ACM, 2012: 119-122.
[32]Hanna GB, Shimi SM, Cuschieri A. Task performance in endoscopic surgery is influenced by location of the image display [J]. Annals of Surgery, 1998, 227(4): 481-484.
[33]李舫, 宋志坚. HMD式光学穿透技术在医学增强现实中的研究进展 [J]. 中国数字医学, 2012, 7(1): 14-20.
[34]Kellner F, Bolte B, Bruder G, et al. Geometric calibration of headmounted displays and its effects on distance estimation [J]. IEEE Transactions on Visualization and Computer Graphics, 2012, 18(4): 589-596.