Cross-Subjects EEG Emotion Recognition Based on Scaled Convolutional Attention Network
Chen Binbin1, Wu Tao1,2, Chen Lifei1,2,3*
1(School of Computer and Cyberspace Security, Fujian Normal University, Fuzhou 350117, China) 2(Digital Fujian Environmental Monitoring Internet of Things Laboratory, Fujian Normal University, Fuzhou 350117, China) 3(Fujian Provincial Center of Applied Mathematics, Fujian Normal University, Fuzhou 350117, China)
Abstract Emotion recognition based on EEG signals has become a medical aid to emotion regulation intervention because it can objectively reflect human physiological and psychological states. To address the problem of poor generalization performance of emotion recognition caused by existing methods ignoring the differences in channel data distribution between individuals, a new cross-subjects emotion recognition method based on scaled convolutional attention network was proposed in this work. Based on the extraction of emotion quantification features in multi-channel electroencephalography (EEG) signals, a novel scaled convolutional attention network was constructed to establish the synergistic change relationship of emotional features of different channels and scales, and the weight of the synergistic relationship was automatically learned by model training, and finally the domain invariant representation of the emotion polarity was obtained, which improved the generalization performance of emotion recognition across multiple individuals EEG. The emotion EEG dataset SEED and SEED-IV were used to identify emotions cross-subjects with100 665 and 100 950 EEG samples, and the recognition accuracy of the proposed method was 89.63% and 75.65% in the three and four-class of emotions. Particularly, the robustness of the proposed model outperformed most of the existing methods when there were changes in the number of individuals. The results showed that the proposed method was able to effectively extract the domain-invariant representation of emotional polarity.
[1] Egger M, Ley M, Hanke S. Emotion recognition from physiological signal analysis: a review[J]. Electronic Notes in Theoretical Computer Science,2019,343: 35-55.
[2] Ehring T,Fische S, Schnülle J, et al. Characteristics of emotion regulation in recovered depressed versus never depressed individuals[J]. Personality and Individual Differences,2008,44(7): 1574-1584.
[3] 王婷,朱卓影,徐一峰.广泛性焦虑障碍的情绪调节特征[J].临床精神医学杂志,2021,31(3):241-243.
[4] 王忠民,赵玉鹏,郑镕林,等.脑电信号情绪识别研究综述[J].计算机科学与探索,2022,16(4):760-774.
[5] 周如双,赵慧琳,林玮玥,等.基于深浅特征融合的深度卷积残差网络的脑电情绪识别模型[J].中国生物医学工程学报,2021,40(6):641-652.
[6] 潘家辉,何志鹏,李自娜,等.多模态情绪识别研究综述[J].智能系统学报,2020,15(4): 633-645.
[7] 柴鑫.基于领域适配的跨个体脑电情绪识别方法研究[D].哈尔滨:哈尔滨工业大学,2018.
[8] Zheng Weilong, Lu Baoliang. Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks [J]. IEEE Transactions on Autonomous Mental Development, 2015, 7(3):162-175.
[9] Zhong Meiyu, Yang Qingyu, Liu Yi, et al. EEG emotion recognition based on TQWT-features and hybrid convolutional recurrent neural network [J]. Biomedical Signal Processing and Control, 2023, 79(2): 104211.
[10] Selesnick IW. Wavelet transform with tunable Q-factor[J]. IEEE Transactions on Signal Processing, 2011, 59(8): 3560-3575.
[11] Gosala B, Kapgate PD,Jain P, et al. Wavelet transforms for feature engineering in EEG data processing: An application on Schizophrenia[J]. Biomedical Signal Processing and Control,2023,85: 104811.
[12] Chao Hao, Dong Liang, Liu Yongli, et al. Emotion recognition from multiband EEG signals using capsnet [J]. Sensors, 2019, 19(9):2212.
[13] 李锦瑶,杜肖兵,朱志亮,等.脑电情绪识别的深度学习研究综述[J].软件学报,2023,34(1):255-276.
[14] Pooja, Pahuja SK, Veer Karan. Recent approaches on classification and feature extraction of EEG signal: a review[J]. Robotica,2021,40(1):77-101.
[15] Li Zhongjie, Zhang Gaoyan, Wang Longbiao, et al. Emotion recognition using spatial-temporal EEG features through convolutional graph attention network [J]. Journal of Neural Engineering, 2023, 20(1).
[16] 秦兴彬,颜延,樊建平,等.基于多核超限学习机的实时心电信号分析[J].集成技术,2015,4(5):36-45.
[17] 包广城.基于脑电的跨域情绪识别研究[D].郑州:战略支援部队信息工程大学,2021.
[18] Song Tengfei, Zheng Wenming, Song Peng,et al. EEG emotion recognition using dynamical graph convolutional neural networks[J]. IEEE Transactions on Affective Computing, 2020, 11(3): 532-541.
[19] Zhou Jian, Chu Shujie, Li Xin, et al. An EEG emotion recognition method based on transfer learning and echo state network for HilCPS [J]. Microprocessors and Microsystems, 2021, 87:103381.
[20] Bao Guangcheng, Zhuang Ning, Tong Li, et al. Two-level domain adaptation neural network for EEG-based emotion recognition [J]. Frontiers in Human Neuroscience, 2021, 14: 605246.
[21] Jin Yiming, Luo Yudong, Zheng Weilong, et al. EEG-based emotion recognition using domain adaptation network [C] // 2017 International Conference on Orange Technologies (ICOT). Piscataway:IEEE, 2017: 222-225.
[22] Cao Jiangsheng, He Xueqin, Yang Chenhui, et al. Multi-source and multi-representation adaptation for cross-domain electroencephalography emotion recognition[J]. Front Psychol. 2022,12:809459.
[23] 李景聪,潘伟健,林镇远,等.采用多路图注意力网络的情绪脑电信号识别方法[J].智能系统学报,2022,17(3):531-539.
[24] Javier F, Nicholas G, Olaf W, et al. Cross-subject EEG-based emotion recognition through neural networks with stratified normalization [J]. Frontiers in Neuroscience,2021,15:626277.
[25] Chen Hao, Jin Ming, Li Zhunan, et al. MS-MDA: Multisource marginal distribution adaptation for cross-subject and cross-cession EEG emotion recognition [J]. Frontiers in Neuroscience, 2021,15:778488.
[26] 蓝文威,陈晨,张金,等.联合脑电信号与虚拟技术的大脑情绪状态的半球不对称性研究[J].中国生物医学工程学报,2021,40(3):266-271.
[27] 王晨,胡景钊,刘科,等.基于脑电通道增强的情绪识别方法[J].西北大学学报(自然科学版),2022,52(4):560-570.
[28] 李志鹏.情感脑电的通道选择与分类方法研究[D].哈尔滨:哈尔滨工业大学,2017.
[29] Zheng Weilong, Lu Baoliang. Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks[J].IEEE Transactions on Autonomous Mental Development (IEEE TAMD),2015,7(3): 162-175.
[30] Zheng Weilong, Liu Wei, Lu Yifei, et al. EmotionMeter: a multimodal framework for recognizing human emotions[J]. IEEE Transactions on Cybernetics, 2019, 49(3): 1110-1122.
[31] Krishna AH, Sri Ab, Priyanka KYVS, et al. Emotion classification using EEG signals based on tunable- Q wavelet transform [J]. IET Science, Measurement & Technology, 2019, 13(3): 375-380.
[32] 张俊杰, 郝李刚, 许茜,等. 基于临床及CT特征构建预测肺浸润性黏液腺癌的机器学习模型[J]. 中华全科医学, 2023, 21(1): 6-9.
[33] 赵越,曾立波,吴琼水.卷积神经网络的宫颈细胞图像分类[J].计算机辅助设计与图形学学报,2018(11):2049-2054.
[34] Wang Qilong, Wu Banggu, Zhu Pengfei, et,al. ECA-Net: Efficient channel attention for deep convolutional neural networks[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle: IEEE, 2020: 11531-11539.
[35] 郑伟龙,石振锋,吕宝粮.用异质迁移学习构建跨被试脑电情感模型[J].计算机学报,2020,43(2):177-189.
[36] Wang Yixin, Qiu Shuang, Li Dan, et al. Multi-modal domain adaptation variational autoencoder for EEG-based emotion recognition[J]. IEEE/ CAA Journal of Automatica Sinica, 2022, 9(9): 1612-1626.
