A Study of Argument Reality Based Brain-Computer Interface (AR-BCI) in Hololens
Zhang Lixin1, Zhang Yukun1, Ke Yufeng2#*, Du Jiale1, Xu Minpeng1#, Ming Dong1,2#
1School of Precision Instrument and Optoelectronics Engineering, Tianjin University, Tianjin 300072, China; 2Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin 300072, China
Abstract：Brain-Computer interfaces (BCIs) have improved greatly in the last decades. However, high-performance BCIs usually need display devices to present the visual stimulus to evoke specific EEG patterns. The most popular display device now is computer monitor that is not portable and thus restrict the portability of BCI. By combining argument reality (AR) technology with BCIs, the problem can be resolved, achieving a more practically applicable BCI system. Currently, how to raise the speed and accuracy of AR-BCI remains an open question. This study proposed an AR-BCI system based on Hololens. We generated eight stimuli in the argument reality environment to evoke different steady state visual evoke potentials (SSVEP). Twelve subjects participated in this study and their SSVEP patterns were successfully evoked in AR environment. They achieved 88.67% and 98.6% accuracy on average for SSVEP-BCI with EEG data length of 1 s and 2 s respectively. Our Result showed that AR-BCI is promising to achieve a high-performance portable and wearable control system in daily life.
作者简介: #中国生物医学工程学会会员(Member, Chinese Society of Biomedical Engineering)
引用本文:
张力新, 张裕坤, 柯余峰, 杜佳乐, 许敏鹏, 明东. 基于Hololens的增强现实脑-机接口研究[J]. 中国生物医学工程学报, 2019, 38(1): 51-58.
Zhang Lixin, Zhang Yukun, Ke Yufeng, Du Jiale, Xu Minpeng, Ming Dong. A Study of Argument Reality Based Brain-Computer Interface (AR-BCI) in Hololens. Chinese Journal of Biomedical Engineering, 2019, 38(1): 51-58.
[1] Wolpaw JR, Birbaumer N, Heetderks WJ, et al. Brain-computer interface technology: A review of the first international meeting [J]. IEEE Trans Rehabil Eng, 2000, 8(2): 164-173. [2] Herrmann CS. Human EEG responses to 1-100 Hz flicker: resonance phenomena in visual cortex and their potential correlation to cognitive phenomena [J]. Experimental Brain Research, 2001, 137(3-4): 346-353. [3] Nakanishi M, Wang YJ, Chen XG, et al. Enhancing detection of SSVEPs for a high-speed brain speller using task-related component analysis [J]. IEEE Transactions on Biomedical Engineering, 2018, 65(1): 104-112. [4] Chen X, Wang Y, Nakanishi M, et al. High-speed spelling with a noninvasive brain-computer interface [J]. Proceedings of the National Academy of Sciences, 2015, 112(44): E6058-E6067. [5] Zhang Y, Zhou G, Jin J, et al. Frequency recognition in SSVEP-based BCI using multiset canonical correlation analysis [J]. International Journal of Neural Systems, 2014, 24(4): 1450013. [6] Wang H, Zhang Y, Waytowich NR, et al. Discriminative feature extraction via multivariate linear regression for SSVEP-based BCI [J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2016, 24(5): 532-541. [7] Krucoff MO, Rahimpour S, Slutzky MW, et al. Enhancing nervous system recovery through neurobiologics, neural interface training, and neurorehabilitation [J]. Frontiers in Neuroscience, 2016, 10:584. [8] Kansaku K, Hata N, Takano K. My thoughts through a robot′s eyes: An augmented reality-brain-machine interface [J]. Neuroscience Research, 2010, 66(2): 219-222. [9] Lenhardt A, Ritter H. An augmented-reality based brain-computer interface for robot control [C] //Neural Onformation Processing. Sydney: Asia Pacific Neural Network Assembly, 2010: 58-65. [10] Takano K, Hata N, Kansaku K. Towards intelligent environments: an augmented reality-brain-machine interface operated with a see-through head-mount display [J]. Frontiers in Neuroscience, 2011, 5:60. [11] Horii S, Nakauchi S, Kitazaki M. AR-SSVEP for brain-machine interface: Estimating user′s gaze in head-mounted display with USB camera [C] // Proceedings of the IEEE Virtual Reality Annual International Symposium. Paris: IEEE, 2015: 193-194. [12] Faller J, Allison BZ, Brunner C, et al. A feasibility study on SSVEP-based interaction with motivating and immersive virtual and augmented reality [J]. arXiv:1701.03981, 2017. [13] Kapeller C, Hintermüller C, Guger C. Augmented control of an avatar using an SSVEP based BCI [C] //Proceedings of the 3rd Augmented Human International Conference. Paris: ACM, 2012: 27. [14] Kerous B, Liarokapis F. BrainChat-A collaborative augmented reality brain interface for message communication [C] // Proceedings of the Mixed and Augmented Reality. Paris: IEEE, 2017: 279-283 [15] Martens N, Jenke R, Abu-Alqumsan M, et al. Towards robotic re-embodiment using a brain-and-body-computer interface [C] // Intelligent Robots and Systems. Portugal: IEEE, 2012: 5131-5131. [16] Paszkiel S. Augmented reality of technological environment in correlation with brain computer interfaces for control processes [M] // Recent Advances in Automation, Robotics and Measuring Techniques. Beilin:Springer, 2014: 197-203. [17] Petit D, Gergondet P, Cherubini A, et al. Navigation assistance for a bci-controlled humanoid robot[C] //The 4th Annual IEEE International Conference on Cyber Technology in Automation, Control and Intelligent. Hong Kong: IEEE, 2014:246-251. [18] Chen X, Wang Y, Gao S, et al. Filter bank canonical correlation analysis for implementing a high-speed SSVEP-based brain-computer interface [J]. Journal of Neural Engineering, 2015, 12(4): 046008. [19] McFarland DJ, Sarnacki WA, Wolpaw JR. Brain-computer interface (BCI) operation: Optimizing information transfer rates [J]. Biological Psychology, 2003, 63(3): 237-251. [20] Yuan P, Gao X, Allison B. A study of the existing problems of estimating the information transfer rate in online brain-computer interfaces [J]. Journal of Neural Engineering, 2013, 10(2): 026014.
