Review of Researches on Common Spatial Pattern and its Extended Algorithms for MovementIntention Decoding
Pan Lincong1, Wang Kun1, Xu Minpeng1,2, Ni Guangjian1,2, Ming Dong1,2
1(Academy of Medical Engineering and Translational Medicine. Tianjin University, Tianjin 300072, China) 2(School of Precision Instrument and Opto-electronics Engineering, Tianjin University, Tianjin 300072, China)
Abstract：Motor imagery based-brain-computer interfaces (MI-BCIs) are of important research significance and application value in rehabilitation, replacement, and enhancement of human motor function. Common spatial pattern (CSP) algorithm aims to enhance the difference of electroencephalography (EEG) features induced by MI, which is currently one of the most widely used feature extraction algorithms for MI paradigm. However, it does not focus on the time and frequency domain information of EEG, and is sensitive to noise and deviation values, resulting in the limited recognition performance and the low robustness of classifiers. This paper reviewed the development history of CSP and its extended algorithms. We introduced the basic principles and key calculation steps of representative extended algorithms in detail from three aspects: multi-modal information optimization, regularization optimization and other spatial mapping optimization methods. In addition, we discussed the actual challenges and predict the future development trend, aiming to promote the in-depth research and application of relevant BCI technology.
潘林聪, 王坤, 许敏鹏, 倪广健, 明东. 面向运动意图解码的共空间模式及其扩展算法研究综述[J]. 中国生物医学工程学报, 2022, 41(5): 577-588.
Pan Lincong,Wang Kun,Xu Minpeng,Ni Guangjian,Ming Dong,. Review of Researches on Common Spatial Pattern and its Extended Algorithms for MovementIntention Decoding. Chinese Journal of Biomedical Engineering, 2022, 41(5): 577-588.
[1] Zhou Maigeng, Wang Haidong, Zeng Xinying, et al. Mortality, morbidity, and risk factors in China and its provinces, 1990-2017: a systematic analysis for the Global Burden of Disease Study 2017 [J]. Lancet, 2019, 394(10204): 1145-1158. [2] Ramadan RA, Vasilakos AV. Brain computer interface: control signals review [J]. Neurocomputing, 2017, 223: 26-44. [3] Juliano JM, Spicer RP, Vourvopoulos A, et al. Embodiment is related to better performance on a brain-computer interface in immersive virtual reality: a pilot study [J]. Sensors, 2020, 20(4): 1204-1221. [4] Chen Xiaogang, Huang Xiaoshan, Wang Yijun, et al. Combination of augmented reality based brain-computer interface and computer vision for high-level control of a robotic arm [J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2020, 28(12): 3140-3147. [5] Liang WD, Xu Y, Schmidt J, et al. Upregulating excitability of corticospinal pathways in stroke patients using TMS neurofeedback; A pilot study [J]. Neuroimage-Clinical, 2020, 28(102465):1-8. [6] Molla MKI, Saha SK, Yasmin S, et al. Trial regeneration with subband signals for motor imagery classification in BCI paradigm [J]. IEEE Access, 2021, 9: 7632-7642. [7] Benzy VK, Vinod AP, Subasree R, et al. Motor imagery hand movement direction decoding using brain computer interface to aid stroke recovery and rehabilitation [J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2020, 28(12): 3051-3062. [8] Kales ZJ, Lazar MS, Zhou SZ. Spatial patterns underlying population differences in the background EEG [J]. Brain Topography, 1990, 2(4): 275-284. [9] Jin Jing, Xiao Ruocheng, Daly I, et al. Internal feature selection method of CSP based on l1-norm and dempster-Shafer theory [J]. IEEE Transactions on Neural Networks and Learning Systems, 2020, 99: 1-12. [10] Zhang Ce, Eskandarian A. A computationally efficient multiclass time-frequency common spatial pattern analysis on EEG motor imagery [C] // The 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). Montreal: IEEE, 2020: 514-518. [11] Talukdar U, Hazarika SM, Gan JQ. Adaptation of common spatial patterns based on mental fatigue for motor-imagery BCI [J]. Biomedical Signal Processing and Control, 2020, 58: 101829. [12] Zhang Yuan, Guo Yao, Yang Po, et al. Epilepsy seizure prediction on EEG using common spatial pattern and convolutional neural network [J]. IEEE Journal of Biomedical and Health Informatics, 2020, 24(2): 465-474. [13] 王玉潇,姜威,刘治,等. 基于共空间模式算法和支持向量机二重分类的癫痫发病预测 [J]. 生物医学工程学杂志, 2021, 38(1): 39-46. [14] 陈景霞,郑茹,张鹏伟,等. 基于数据空间自适应与共空间模式的脑电情感分类 [J]. 计算机工程, 2019, 45(4): 296-301. [15] Jiang Chao, Li Yingjie, Tang Yingying, et al. Enhancing EEG-Based Classification of Depression Patients Using Spatial Information [J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2021, 29: 566-575. [16] 王斐,王少楠,王惜慧,等. 基于脑电图识别结合操纵特征的驾驶疲劳检测 [J]. 仪器仪表学报, 2014, 35(2): 398-404. [17] Zeng Hong, Yang Chen, Zhang Hua, et al. A LightGBM-based EEG analysis method for driver mental states classification [J]. Computational Intelligence and Neuroscience, 2019, 3761203: 1-11. [18] Shapiro J, Savransky D, Ruffio JB, et al. Detecting planets from direct-imaging observations using common spatial pattern filtering [J]. Astronomical Journal, 2019, 158(3): 1-14. [19] Talukdar U, Hazarika SM, Gan JQ. Adaptive feature extraction in EEG-based motor imagery BCI: tracking mental fatigue [J]. Journal of Neural Engineering, 2020, 17(1), 016020. [20] Deng Yue, Li Zhanli, Wang Haixian, et al. Local temporal joint recurrence common spatial patterns for mi-based BCI [C] // The 4th Information Technology, Networking, Electronic and Automation Control Conference(ITNEC). Chongqing: IEEE, 2020: 813-816. [21] Lotte F, Guan C. Spatialy regularized common spatial patterns for EEG classification [C] // The 20th International Conference on Pattern Recognition. Istanbul: IEEE, 2010: 3712-3715. [22] Lemm S, Blankertz B, Curio G, et al. Spatio-spectral filters for improving the classification of single trial EEG [J]. IEEE Transactions on Biomedical Engineering, 2005, 52(9): 1541-1548. [23] Dornhege G, Blankertz B, Krauledat M, et al. Combined optimization of spatial and temporal filters for improving brain-computer interfacing [J]. IEEE Transactions on Biomedical Engineering, 2006, 53(11): 2274-2281. [24] Blankertz B, Kawanabe M, Tomioka R, et al. Invariant common spatial patterns: Alleviating nonstationarities in brain-computer interfacing [M] //Advances in Neural Information Processing Systems 20. Vancouver: MIT Press, 2007: 113-120. [25] Ang KK, Chin ZY, Wang C, et al. Filter bank common spatial pattern algorithm on BCI competition IV datasets 2a and 2b [J]. Frontiers in Neuroscience, 2012, 6(39):1-9. [26] Thomas KP, Guan C, Tong LC, et al. An adaptive filter bank for motor imagery based brain computer interface [C] // The 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Vancouver: IEEE, 2008: 1104-1107. [27] Thomas KP, Guan C, Tong LC, et al. Discriminative FilterBank Selection and EEG information Fusion for Brain Computer Interface [C] //International Symposium on Circuits and Systems. Taipei: IEEE, 2009: 1469-1472. [28] Thomas KP, Guan C, Tong LC, et al. A new discriminative common spatial pattern method for motor imagery brain-computer interfaces [J]. IEEE Transactions on Biomedical Engineering, 2009, 56(11): 2730-2733. [29] Hyohyeong K, Yunjun N, Seungjin C. Composite Common Spatial Pattern for Subject-to-Subject Transfer [J]. IEEE Signal Processing Letters, 2009, 16(8): 683-686. [30] Lu Haiping, Plataniotis KN, Venetsanopoulos AN. Regularized common spatial patterns with generic learning for EEG signal classification [C] //Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Minneapolis: IEEE, 2009:6599-6602. [31] Lotte F, Guan C. Regularizing common spatial patterns to improve BCI designs: unified theory and new algorithms [J]. IEEE Transactions on Biomedical Engineering, 2011, 58(2): 355-362. [32] Barachant A, Bonnet S, Congedo M, et al. Common spatial pattern revisited by Riemannian geometry [C] //IEEE International Workshop on Multimedia Signal Processing (MMSP). Saint-Malo: IEEE, 2010: 472-476. [33] Mousavi EA, Maller JJ, Fitzgerald PB, et al. Wavelet common spatial pattern in asynchronous offline brain computer interfaces [J]. Biomedical Signal Processing and Control, 2011, 6(2): 121-128. [34] Arvaneh M, Guan C, Ang KK, et al. Spatially sparsed Common Spatial Pattern to improve BCI performance [C] //IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Prague: IEEE, 2011: 2412-2415. [35] Albalawi H, Song X. A study of kernel CSP-based motor imagery brain computer interface classification [C] //Signal Processing in Medicine and Biology Symposium(SPMB). New York: IEEE, 2012: 1-4. [36] Zhang Rui, Xu Peng, Liu Tiejun, et al. Local temporal correlation common spatial patterns for single trial EEG classification during motor imagery [J]. Computational and Mathematical Methods in Medicine, 2013, 2013(591216):1-7. [37] Zhang Yu, Zhou Guoxu, Jin Jing, et al. Optimizing spatial patterns with sparse filter bands for motor-imagery based brain-computer interface [J]. Journal of Neuroscience Methods, 2015, 255: 85-91. [38] Zhang Yu, Wang Yu, Jin Jing, et al. Sparse bayesian learning for obtaining sparsity of EEG frequency bands based feature vectors in motor imagery classification [J]. International Journal of Neural Systems, 2017, 27(2): 1650032, 1-13. [39] Kumar S, Mamun K, Sharma A. CSP-TSM: Optimizing the performance of Riemannian tangent space mapping using common spatial pattern for MI-BCI [J]. Computers in Biology and Medicine, 2017, 91: 231-242. [40] Dai Mengxi, Zheng Dezhi, Liu Shucong, et al. Transfer kernel common spatial patterns for motor imagery brain-computer interface classification [J]. Computational and Mathematical Methods in Medicine, 2018, 2018(9871603):1-9. [41] Zhang Yu, Nam CS, Zhou Guoxu, et al. Temporally constrained sparse group spatial patterns for motor imagery BCI [J]. IEEE Transactions on Cybernetics, 2019, 49(9): 3322-3332. [42] Park Y, Chung W. Frequency-optimized local region common spatial pattern approach for motor imagery classification [J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2019, 27(7): 1378-1388. [43] Jin Jing, Miao Yangyang, Daly I, et al. Correlation-based channel selection and regularized feature optimization for MI-based BCI [J]. Neural Networks, 2019, 118: 262-270. [44] Chakraborty B, Ghosh L, Konar A. Designing phase-sensitive common spatial pattern filter to improve brain-computer interfacing [J]. IEEE Transactions on Biomedical Engineering, 2020, 67(7): 2064-2072. [45] Azab AM, Ahmadi H, Mihaylova L, et al. Dynamic time warping-based transfer learning for improving common spatial patterns in brain-computer interface [J]. Journal of Neural Engineering, 2020, 17(1), 016061:1-10. [46] Guo Yao, Zhang Yuan, Chen Zhiqiang, et al. EEG classification by filter band component regularized common spatial pattern for motor imagery [J]. Biomedical Signal Processing and Control, 2020, 59, 101917: 1-9. [47] Hjorth B, Rodin E. An eigenfunction approach to the inverse problem of EEG [J]. Brain Topogr, 1988, 1(2): 79-86. [48] Müller-Gerking J, Pfurtscheller G, Flyvbjerg H. Designing optimal spatial filters for single-trial EEG classification in a movement task [J]. Clinical Neurophysiology, 1999, 110(5): 787-798. [49] Ramoser H, Müller-Gerking J, Pfurtscheller G. Optimal spatial filtering of single trial EEG during imagined hand movement [J]. IEEE Transactions on Rehabilitation Engineering, 2000, 8(4): 441-446. [50] Wang Yunhua, Berg P, Scherg M. Common spatial subspace decomposition applied to analysis of brain responses under multiple task conditions: a simulation study [J]. Clinical Neurophysiology, 1999, 110(4): 604-614. [51] 张力新, 张珊珊, 王坤, 等. 运动相关思维诱发脑电信息解码与应用综述 [J]. 仪器仪表学报, 2019, 40(1): 1-11. [52] Wang Haixian, Zheng Wenming. Local temporal common spatial patterns for robust single-trial EEG classification [J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2008, 16(2): 131-139. [53] Novi Q, Guan C, Dat TH, et al. Sub-band common spatial pattern (SBCSP) for brain-computer interface [C] // The 3rd International IEEE/EMBS Conference on Neural Engineering. Hawaii: IEEE, 2007: 204-207. [54] Ang KK, Chin ZY, Zhang Haihong, et al. Robust filter bank common spatial pattern (RFBCSP) in motor-imagery-based brain-computer interface [C] // The 31st Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Minneapolis: IEEE, 2009: 578-581. [55] Sun Gufei, Hu Jinglu, Wu Gengfeng. A novel frequency band selection method for common spatial pattern in motor imagery based brain computer interface [C] //International Joint Conference on Neural Networks (IJCNN). Barcelona: IEEE, 2010: 1-6. [56] Yang Banghua, Li Huarong, Wang Qian, et al. Subject-based feature extraction by using fisher WPD-CSP in brain-computer interfaces [J]. Computer Methods and Programs in Biomedicine, 2016, 129: 21-28. [57] He Wei, Wei Pengfei, Wang Liping, et al. A novel EMD-based common spatial pattern for motor imagery brain-computer interface [C] //Proceedings of 2012 IEEE-EMBS International Conference on Biomedical and Health Informatics. Hong Kong: IEEE, 2012: 216-219. [58] Tomioka R, Dornhege G, Nolte G, et al. Spectrally weighted common spatial pattern algorithm for single trial EEG classification [J]. Mathematical Engineering Technical Reports, 2006,113, 8656:1-23. [59] Zhao Qibin, Zhang Liqing, Cichocki A. Multilinear generalization of common spatial pattern [C] //Proceedings of the IEEE International Conference on Acoustics (ICASSP). Taipei: IEEE, 2009: 525-528. [60] Wang Yijun, Gao Shangkai, Gao Xiaorong. Common spatial pattern method for channel selection in motor imagery based brain-computer interface [C] // The 27th Annual Conference of IEEE EMBS. Shanghai: IEEE, 2005: 356-359. [61] Meng Jianjun, Liu Guangquan, Huang Gan, et al. Automated selecting subset of channels based on CSP in motor imagery brain-computer interface system [C] //IEEE International Conference on Robotics and Biomimetics (ROBIO). Guilin: IEEE, 2009: 2290-2294. [62] Li Mingai, Ma Jianyong, Jia Songmin. Optimal combination of channels selection based on common spatial pattern algorithm [C] //International Conference on Mechatronics and Automation. Beijing: IEEE, 2011: 295-300. [63] Park Y, Chung W. Optimal channel selection using correlation coefficient for CSP based EEG classification [J]. IEEE Access, 2020, 8: 111514-111521. [64] Varsehi H, Firoozabadi SMP. An EEG channel selection method for motor imagery based brain-computer interface and neurofeedback using Granger causality [J]. Neural Networks, 2021, 133: 193-206. [65] Sannelli C, Vidaurre C, Müller KR, et al. Common spatial pattern patches - an optimized filter ensemble for adaptive brain-computer interfaces [C] // The 32nd Annual International Conference of the IEEE Engineering in Medicine and Biology. Buenos Aires: IEEE, 2010: 4351-4354. [66] Park Y, Chung W. Filter-bank local region complex-valued common spatial pattern for Motor imagery classification [C] // The 8th International Winter Conference on Brain-Computer Interface (BCI). Gangwon: IEEE, 2020: 138-142. [67] Jiang Jing, Wang Chunhui, Wu Jinghan, et al. Temporal combination pattern optimization based on feature selection method for motor imagery BCIs [J]. Frontiers in Human Neuroscience, 2020, 14(231):1-11. [68] Higashi H, Tanaka T. Common spatio-time-frequency patterns for motor imagery-based brain machine interfaces [J]. Computational Intelligence and Neuroscience, 2013, 2013(537218):1-13. [69] Jiao Yong, Zhou Tao, Yao Lina, et al. Multi-view multi-scale optimization of feature representation for EEG classification improvement [J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2020, 28(12): 2589-2597. [70] Cho H, Ahn M, Ahn S, et al. Invariant common spatio-spectral patterns [C] // The 3rd TOBI Workshop. Würzburg: TOBI, 2012: 31-32. [71] Hatamikia S, Nasrabadi A M. Subject transfer BCI based on Composite Local Temporal Correlation Common Spatial Pattern [J]. Comput Biol Med, 2015, 64: 1-11. [72] Lu Haiping, Eng HL, Guan C, et al. Regularized common spatial pattern with aggregation for EEG classification in small-sample setting [J]. IEEE Transactions on Biomedical Engineering, 2010, 57(12): 2936-2946. [73] Park SH, Lee D, Lee SG. Filter bank regularized common spatial pattern ensemble for small sample motor imagery classification [J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2018, 26(2): 498-505. [74] 周晓宇, 许敏鹏, 肖晓琳,等. 脑-机接口中脑电解码算法研究综述[J].生物医学工程学杂志,2019,36(5): 856-861. [75] Congedo M, Barachant A, Bhatia R. Riemannian geometry for EEG-based brain-computer interfaces; a primer and a review [J]. Brain-Computer Interfaces, 2017, 4(3): 155-174. [76] Fletcher PT, Joshi S. Principal geodesic analysis on symmetric spaces: statistics of diffusion tensors [C] //Computer Vision and Mathematical Methods in Medical and Biomedical Image Analysis. Berlin: Springer, 2004: 87-98. [77] Barachant A, Bonnet S, Congedo M, et al. Multiclass brain-computer interface classification by Riemannian geometry [J]. IEEE Transactions on Biomedical Engineering, 2012, 59(4): 920-928. [78] Chen Zhihua, Zhou Hong, Zhao Li. Decoding human right and left hand motor imagery from EEG single trials using sample entropy [C] //International Conference on Electronics and Optoelectronics (ICEOE). Dalian: IEEE, 2011, 4: 353-356. [79] Wang Lei, Xu Guizhi, Yang Shuo, et al. Motor imagery BCI research based on sample entropy and SVM [C] //The 6th International Conference on Electromagnetic Field Problems and Applications. Dalian: IEEE, 2012: 1-4. [80] Chen Shunfei, Luo Zhizeng, Gan Haitao. An entropy fusion method for feature extraction of EEG [J]. Neural Computing and Applications, 2016, 29(10): 857-863. [81] Sharma R, Pachori R, Acharya U. Application of entropy measures on intrinsic mode functions for the automated identification of focal electroencephalogram signals [J]. Entropy, 2015, 17(2): 669-691. [82] 吴小培,周蚌艳,张磊,等. 脑-机接口中空域滤波技术现状与进展 [J]. 安徽大学学报(自然科学版), 2017, 41(2): 14-31. [83] Tayeb Z, Fedjaev J, Ghaboosi N, et al. Validating deep neural networks for online decoding of motor imagery movements from EEG signals [J]. Sensors, 2019, 19(1): 210-226. [84] Jiang Qin, Zhang Yi, Ge Gengyu, et al. An adaptive CSP and clustering classification for online motor imagery EEG [J]. IEEE Access, 2020, 8: 156117-156128.
