Research Advances in Deep Learning-Based Multimodal Epilepsy Detection and Seizure Prediction
Yu Jian1,2, Deng Qishichao1,2, Wu Jiacheng1,2, Li Chuantao3, Lin Dongmei4, Yan Zuojian2, Chen Fuming2*
1(School of Medical Information Engineering, Gansu University of Chinese Medicine, Lanzhou 730000, China) 2(Department of Medical Engineering, 940th Hospital of Joint Logistics Support Force of Chinese People′s Liberation Army, Lanzhou 730050, China) 3(Naval Characteristic Medical Center, Naval Medical University (Second Military Medical University) Shanghai 200433, China) 4(School of Electrical and Information Engineering, Lanzhou University of Technology, Lanzhou 730050, China)
Abstract：Epilepsy is a chronic neurological disorder caused by abnormal synchronous neuronal discharge in the brain, affecting more than 70 million people worldwide. Accurate detection and seizure prediction of epilepsy are of critical clinical significance for reducing the risk of sudden unexpected death in epilepsy and improving patients' quality of life. Current mainstream methods mostly rely on single electroencephalogram (EEG) signals, which can hardly comprehensively capture the multi-system pathological changes accompanying epileptic seizures. This article reviewed the research progress in deep learning-based multimodal epilepsy detection and seizure prediction. First, commonly used physiological signals were summarized, including electroencephalogram (EEG), electrocardiogram (ECG), accelerometer and electrodermal activity, as well as public multimodal datasets such as EPILEPSIAE and TUSZ. Then, the principles, advantages, limitations and applicable scenarios of three core fusion strategies (early fusion, late fusion and hybrid fusion) were compared. The technical characteristics and performance of convolutional neural networks (CNN), long short-term memory networks (LSTM) and hybrid deep learning models were elaborated, and the mainstream technical route that takes EEG as the core and complements with other multimodal physiological signals were clarified. Finally, core challenges were pointed out, such as insufficient cross-patient generalization ability, difficulties in cross-modal alignment, non-standard data annotation and poor model interpretability. Future research trends were prospected from four aspects: developing robust cross-modal feature fusion algorithms, promoting standardized data construction, enhancing clinical interpretability and realizing lightweight deployment.
俞键, 邓漆时超, 吴佳成, 李川涛, 林冬梅闫作剑, 陈扶明. 基于深度学习的多模态癫痫检测及发作预测研究进展[J]. 中国生物医学工程学报, 2026, 45(3): 362-371.
Yu Jian, Deng Qishichao, Wu Jiacheng, Li Chuantao, Lin Dongmei, Yan Zuojian, Chen Fuming. Research Advances in Deep Learning-Based Multimodal Epilepsy Detection and Seizure Prediction. Chinese Journal of Biomedical Engineering, 2026, 45(3): 362-371.
[1] Moshé SL, Perucca E, Ryvlin P, et al. Epilepsy: new advances[J]. The Lancet, 2015, 385(9971): 884-898. [2] Tan S, Tang Z, He Q, et al. Automatic detection and prediction of epileptic EEG signals based on nonlinear dynamics and deep learning: a review[J]. Frontiers in Neuroscience, 2025, 19: 1630664. [3] Klatt J, Feldwisch‐Drentrup H, Ihle M, et al. The EPILEPSIAE database: an extensive electroencephalography database of epilepsy patients[J]. Epilepsia, 2012, 53(9): 1669-1676. [4] Shah V, Von Weltin E, Lopez S, et al. The temple university hospital seizure detection corpus[J]. Frontiers in Neuroinformatics, 2018, 12: 83. [5] Pordoy J, Jones G, Matoorian N, et al. The Open Seizure Database facilitating research into non-EEG seizure detection[EB/OL]. https://www.techrxiv.org/doi/full/10.36227/techrxiv.23957625.v1, 2023-08-16/2026-06-07. DOI:10.36227/techrxiv.23957625.v1. [6] Kuhlmann L, Karoly P, Freestone DR, et al. Epilepsyecosystem.org: crowd-sourcing reproducible seizure prediction with long-term human intracranial EEG[J]. Brain, 2018, 141(9): 2619-2630. [7] Nasseri M, Nurse E, Glasstetter M, et al. Signal quality and patient experience with wearable devices for epilepsy management[J]. Epilepsia, 2020, 61(S1): S25-S35. [8] Bhagubai M, Chatzichristos C, Swinnen L, et al. SeizeIT2: wearable dataset of patients with focal epilepsy[J]. Scientific Data, 2025, 12(1): 1228. [9] Nasreddine W. Epileptic EEG Dataset [DB/OL]. https://data.mendeley.com/datasets/5pc2j46cbc/1, 2021-03-16/2026-06-07. DOI: 10.17632/5pc2j46cbc.1. [10] Detti P. Siena Scalp EEG Database [DB/OL]. https://physionet.org/content/siena-scalp-eeg/1.0.0/, 2020-08-11/ 2026-06-07. DOI: 10.13026/5d4a-j060. [11] Baltrušaitis T, Ahuja C, Morency L P. Multimodal machine learning: a survey and taxonomy[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 41(2): 423-443. [12] Stahlschmidt SR, Ulfenborg B, Synnergren J. Multimodal deep learning for biomedical data fusion: a review[J]. Briefings in Bioinformatics, 2022, 23(2): bbab569. [13] Karasmanoglou A, Giannakakis G, Vorgia P, et al. Semi-supervised anomaly detection for the prediction and detection of pediatric focal epileptic seizures on fused EEG and ECG data[J]. Biomedical Signal Processing and Control, 2025, 101: 107083. [14] Zambrana-Vinaroz D, Vicente-Samper JM, Manrique-Cordoba J, et al. Wearable epileptic seizure prediction system based on machine learning techniques using ECG, PPG and EEG signals[J]. Sensors, 2022, 22(23): 9372. [15] Li S, Tang H. Multimodal alignment and fusion: a survey[J]. International Journal of Computer Vision, 2026, 134(3): 103. [16] Kumar S, Rani S, Sharma S, et al. Multimodality fusion aspects of medical diagnosis: a comprehensive review[J]. Bioengineering, 2024, 11(12): 1233. [17] Chango W, Lara JA, Cerezo R, et al. A review on data fusion in multimodal learning analytics and educational data mining[J]. WIREs Data Mining and Knowledge Discovery, 2022, 12(4): e1458. [18] Zhang J, Swinnen L, Chatzichristos C, et al. Multimodal wearable EEG, EMG and accelerometry measurements improve the accuracy of tonic-clonic seizure detection[J]. Physiological Measurement, 2024, 45(6): 065005. [19] Vandecasteele K, De Cooman T, Chatzichristos C, et al. The power of ECG in multimodal patient‐specific seizure monitoring: added value to an EEG‐based detector using limited channels[J]. Epilepsia, 2021, 62(10): 2333-2343. [20] Cui C, Yang H, Wang Y, et al. Deep multimodal fusion of image and non-image data in disease diagnosis and prognosis: a review[J]. Progress in Biomedical Engineering, 2023, 5(2): 022001. [21] Zhao F, Zhang C, Geng B. Deep multimodal data fusion[J]. ACM Computing Surveys, 2024, 56(9): 1-36. [22] Jiao T, Guo C, Feng X, et al. A comprehensive survey on deep learning multi-modal fusion: methods, technologies and applications[J]. Computers, Materials & Continua, 2024, 80(1): 1-35. [23] Khalid AM, Sulemana M, Abdul IW. Equation-level parameterized fusion reformulation for multimodal epileptic seizure detection using interaction control and data-quality screening[J]. Frontiers in Signal Processing, 2026, 6: 1745291. [24] Mamatha AS, Sooda K. Advancements in epilepsy classification: current trends and future directions[J]. MethodsX, 2025, 14: 103257. [25] Liu Y, Sivathamboo S, Goodin P, et al. Epileptic seizure detection using convolutional neural network: a multi-biosignal study [C]//Proceedings of the Australasian Computer Science Week Multiconference. Melbourne: ACM, 2020: 1-8. [26] Tang J, El Atrache R, Yu S, et al. Seizure detection using wearable sensors and machine learning: setting a benchmark[J]. Epilepsia, 2021, 62(8): 1807-1819. [27] Ingolfsson TM, Wang X, Chakraborty U, et al. BrainFuseNet: enhancing wearable seizure detection through EEG-PPG-accelerometer sensor fusion and efficient edge deployment[J]. IEEE Transactions on Biomedical Circuits and Systems, 2024, 18(4): 720-733. [28] Seyidova M, Henze J, Pelzer A, et al. Comparison of different data augmentation techniques for improving epileptic seizure detection based on 3D acceleration, heart rate and temperature data [C]//Proceedings of the 17th International Joint Conference on Biomedical Engineering Systems and Technologies. Rome: SCITEPRESS-Science and Technology Publications, 2024: 142-153. [29] Ge Y, Hu D, Cui X, et al. Multimodal wearable device signal based epilepsy detection with multi-scale convolutional neural network [C]//Sun F, Meng Q, Fu Z, et al. Cognitive Systems and Information Processing: Vol. 1919. Singapore: Springer Nature Singapore, 2024: 70-80. [30] Thakare VS, Ranawat R. Multimodality deep learning fusion based epileptic seizure detection[J]. International Journal of Intelligent Engineering and Systems, 2025, 18(2): 679-691. [31] Pordoy J, Jones G, Matoorian N, et al. Enhanced non-EEG multimodal seizure detection: a real-world model for identifying generalised seizures across the ictal state[J]. IEEE Journal of Biomedical and Health Informatics, 2025, 29(5): 3329-3342. [32] Saeizadeh A, Schonholtz D, Neimat JS, et al. A multi-modal non-invasive deep learning framework for progressive prediction of seizures[C]//2024 IEEE 20th International Conference on Body Sensor Networks (BSN). Chicago: IEEE, 2024: 1-4. [33] Saeizadeh A, Schonholtz D, Uvaydov D, et al. SeizNet: an AI-enabled implantable sensor network system for seizure prediction[C]//2024 19th Wireless On-Demand Network Systems and Services Conference (WONS). Chamonix: IEEE, 2024: 57-60. [34] Meisel C, El Atrache R, Jackson M, et al. Machine learning from wristband sensor data for wearable, noninvasive seizure forecasting[J]. Epilepsia, 2020, 61(12): 2653-2666. [35] Nasseri M, Pal Attia T, Joseph B, et al. Ambulatory seizure forecasting with a wrist-worn device using long-short term memory deep learning[J]. Scientific Reports, 2021, 11(1): 21935. [36] Halimeh M, Jackson M, Loddenkemper T, et al. Training size predictably improves machine learning-based epileptic seizure forecasting from wearables[J]. Neuroscience Informatics, 2025, 5(1): 100184. [37] Sabor N, Mohammed H, Li Z, et al. BHI-Net: brain-heart interaction-based deep architectures for epileptic seizures and firing location detection[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2022, 30: 1576-1588. [38] 刘敏. 基于生理信号的癫痫状态识别方法研究 [D]. 济南:山东中医药大学, 2023. [39] Yu S, El Atrache R, Tang J, et al. Artificial intelligence‐enhanced epileptic seizure detection by wearables[J]. Epilepsia, 2023, 64(12): 3213-3226. [40] 马爽. 基于深度学习的癫痫检测模型研究[D]. 临沂:临沂大学, 2025. [41] Dong C, Van Dijk JP, Aarts RM, et al. Detection of nocturnal epileptic seizures using a wearable armband: a deep learning approach combining accelerometry and photoplethysmography signals[J]. Computer Methods and Programs in Biomedicine, 2026, 273: 109087. [42] Vieluf S, Hasija T, Kuschel M, et al. Developing a deep canonical correlation-based technique for seizure prediction[J]. Expert Systems with Applications, 2023, 234: 120986. [43] Yang Y, Qin X, Wen H, et al. Patient-specific approach using data fusion and adversarial training for epileptic seizure prediction[J]. Frontiers in Computational Neuroscience, 2023, 17: 1172987. [44] Ravindranath M, Candan KS, Sapino ML, et al. MMA: metadata supported multi-variate attention for onset detection and prediction[J]. Data Mining and Knowledge Discovery, 2024, 38(4): 1545-1588. [45] Jaddu S, Jaddu S, Gutierrez C, et al. Pre-ictal seizure prediction using personalized deep learning[EB/OL]. https://arxiv.org/abs/2410.05491, 2024-10-07/2026-06-07. DOI: 10.48550/arXiv.2410.05491. [46] Hasija T, Kuschel M, Jackson M, et al. Improving wearable-based seizure prediction by feature fusion using an explainable growing network[J]. Artificial Intelligence in Medicine, 2025, 169: 103228.
