Abstract：Muscle strength is an important parameter reflecting the state of muscles, which can characterize human body's motor function, muscle health and fatigue level. Non-invasive muscle strength assessment technology has significance and wide application value in many fields, such as sports guidance, muscle disease diagnosis, and rehabilitation status evaluation. In this paper, we proposed a muscle strength assessment method based on feature fusion analysis of surface electromyography (sEMG) and ultrasound radiofrequency (RF) signals by deep learning algorithms, which utilized convolutional neural network (CNN) and pooling operations to extract the effective features (CNNFeat) of the signals and serve as inputs to a support vector machine (SVM) classifier for processing and classification. The method explored the ability of CNNFeat to recognize signal features using CNN-SVM network to verify the complementary nature of sEMG and ultrasound RF signal fusion. The sEMG and ultrasound RF signals of the biceps brachii muscle of 10 healthy subjects were collected under different loads, and the processing results in the multi-user scenario mode showed that CNNFeat was able to improve the classification performance with strong robustness compared to the features of conventional EMG and ultrasound RF signals. The accuracy was 84.23% for EMG signals and 89.34% for ultrasound signals, while the accuracy of the fused signals was as high as 96%, and the fused signals have less oscillations and faster loss convergence.
[1] 王成焘. 人体骨肌系统生物力学 [M]. 北京:科学出版社, 2015. [2] 胡广书,汪梦螺.生物医学信号处理研究综述 [J]. 数据采集与处理, 2015, 30(5): 915-932. [3] 刘绍辉. 人体表面肌电信号分析及其在康复医学中的应用 [D]. 长春:长春大学, 2017. [4] 张启宁. 基于肌电信号实现人体上肢运动和力连续估计的方法研究 [D]. 武汉:华中科技大学, 2019. [5] Yu Haibo, Sun Yi, Bai Fengjun et al. A preliminary study of force estimation based on surface EMG: Towards neuromechanically guided soft oral rehabilitation robot [C] // IEEE International Conference on Rehabilitation Robotics. Singapore: IEEE, 2015: 991-996. [6] Masayuki Y, Ryohei K, Masao Y. An evaluation of hand-force prediction using artificial neural-network regression models of surface emg signals for handwear devices [J]. Journal of Sensors, 2017, 2017:1-12. [7] Cao, Hongxin, Sun, Shouqian, Zhang, Kejun. Modified EMG-based handgrip force prediction using extreme learning machine [J]. Soft Computing, 2017, 21(2): 491-500. [8] Liu Yang, Ning Yong, Li Sheng, et al. Three-dimensional innervation zone imaging from multi-channel surface EMG recordings [J]. International Journal of Neural Systems, 2015, 25(6):1550024. [9] Christophy M, Senan NAF, Lotz JC, et al. A musculoskeletal model for the lumbar spine [J]. Biomechanics and Modeling in Mechanobiology, 2012, 11(1): 19-34. [10] Hägg GM, Luttmann A, Jäger M. Methodologies for evaluating electromyographic field data in ergonomics [J]. Journal of Electromyography and Kinesiology, 2000, 10(5): 301-312. [11] Hoozemans MJM, Van Dieen JH. Prediction of handgrip forces using surface EMG of forearm muscles [J]. Journal of Electromyography and Kinesiology, 2005, 15(4): 358-366. [12] Cerfoglio S, Galli M, Tarabini M, et al. Machine learning-based estimation of ground reaction forces and knee joint kinetics from inertial sensors while performing a vertical drop jump [J]. Sensors, 2021, 21(22): 1-19. [13] Heo P, Gu GM, Lee SJ, et al. Current hand exoskeleton technologies for rehabilitation and assistive engineering [J]. International Journal of Precision Engineering and Manufacturing, 2012, 13(5): 807-824. [14] Nygren AT, Greitz D, Kaijser L. Changes in cross-sectional area in humanexercising and non- exercising skeletal muscles [J]. European Journal of Applied Physiology 2000,81(3): 210-213. [15] Shi Jun, Zheng Yongping, Huang, Qinghua et al. Continuous monitoring of sonomyography, electromyography and torque generated by normal upper arm muscles during isometric contraction: sonomyography assessment for arm muscles [J]. IEEE Transactions on Biomedical Engineering, 2008, 55(3): 1191-1198. [16] Zheng, Yongping, et al. Sonomyography: Monitoring morphological changes of forearm muscles in actions with the feasibility for the control of powered prosthesis [J]. Medical Engineering & Physics, 2006, 28(5): 405-415. [17] Dai Jiayue, Lv Qian, Li Yu, et al. Controllable angle shear wavefront reconstruction based on image fusion method for shear wave elasticity imaging [J]. IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, 2022, 69(1): 187-198. [18] Wang Yuxi, Yin Guanjun, Guo Jianzhong. Evaluation of muscle fatigue state by ultrasonic attenuation coefficient [C] // 2021 IEEE International Ultrasonics Symposium (IUS). Xi'an: IEEE, 2021: 1-4. [19] 王前, 曹霞, 尹冠军,等. 超声图像熵特性的肌肉疲劳进程评估 [J]. 中国生物医学工程学报, 2015, 34(1): 30-36. [20] Li, Pan, Yang Xuebing, Yin Guanjun, et al. Skeletal muscle fatigue state evaluation with ultrasound image entropy [J]. Ultrasonic Imaging, 2020, 42(6): 235-244. [21] 牛英鹏. 用超声图结合肌电图评估肌肉疲劳的方法研究 [J]. 北京体育大学学报, 2008, 31(2): 205-207. [22] Huang, Youjia, Liu, Honghai. Performances of surface EMG andultrasound signals in recognizing finger motion [C] // 2016 9th International Conference on Human System Interactions (HSI). Portsmouth: IEEE, 2016: 117-122. [23] 夏伟. 基于A型超声与表面肌电信号融合的人机接口研究[D]. 上海:上海交通大学, 2019. [24] LeCun Y, Bottou L, Bengio Y, et al. Gradient-based learning applied to document recognition [J]. Proceedings of the IEEE, 1998, 86(11): 2278-2324. [25] Szegedy C, Liu W, Jia Y, et al. Going Deeper with Convolutions [C] // 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston: IEEE, 2014: 1-9. [26] Ren Shaoqing, He Kaiming, Girshick R, et al. Faster R-CNN: towards real-time object detection with region proposal networks [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6):1137-1149. [27] Xu Yong, Du Jun, Dai Lirong, et al. An experimental study on speech enhancement based on deep neural networks [J]. IEEE Signal Processing Letters, 2013, 21(1):65-68. [28] Blumer A, Ehrenfeucht A, Haussler D, et al. Learnability and the Vapnik-Chervonenkis dimension [J]. Journal of the ACM (JACM), 1989, 36(4): 929-965. [29] Burges CJC. A tutorial on support vector machines for pattern recognition [J]. Data Mining and Knowledge Discovery, 1998, 2(2): 121-167. [30] Zhou Yongjin, Li Jizhou, Zhou, Guangquan et al. Dynamic measurement of pennation angle of gastrocnemius muscles during contractions based on ultrasound imaging [J]. Biomedical Engineering Online, 2012, 11(1): 63. [31] 赵汗青. 面向人工假肢的表面肌电信号人手抓取动作研究 [D]. 哈尔滨:哈尔滨理工大学, 2016. [32] 牟永阁. 基于时频和时间尺度分析的表面肌电信号研究及应用 [D].重庆:重庆大学, 2004. [33] Hargrove LJ, Englehart K, Hudgins B. A comparison of surface and intramuscular myoelectric signal classification [J]. IEEE Transactions on Biomedical Engineering, 2007, 54(5): 847-853. [34] Yang Xingchen, Sun, Xueli, Zhou, Dalin, et al. Towards wearable A-mode ultrasound sensing for real-time finger motion recognition [J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2018, 26(6): 1199-1208. [35] 杨亚慧, 谢宏. 基于卷积神经网络的表面肌电信号手势识别 [J]. 微型机与应用, 2017, 36(15): 59-61. [36] Huang Youjia, Yang Xingchen, Li Yuefeng, et al. Ultrasound-based sensing models for finger motion classification [J]. IEEE Journal of Biomedical and Health Informatics, 2018, 22(5): 1395-1405.
