基于无监督深度学习光流法的超声图像运动肌肉分割

doi:10.3969/j.issn.0258-8021.2025.04.006

摘要
图/表
参考文献
相关文章 (15)

全文: PDF (9875 KB) HTML (1 KB)
输出: BibTeX | EndNote (RIS)

摘要准确的图像分割是图像引导治疗脑卒中后痉挛肌肉的基础。本研究提出了基于无监督深度学习的光流算法对运动的靶肌进行分割。由于运动肌肉的超声图像数据集并不具备以往静态超声图像数据集的标签,因而构建了一个无监督光流估计框架。通过增强的自我监督从未标记的图像序列中学习光流,同时为了避免视图合成的目标影响增强后数据的精确性,在转换图像中添加另一个前向传递来扭曲基本学习框架,其中监督来自原始图像的转换预测,最后借助高斯滤波,速度阈值过滤,边框检测过滤和凸包分割等技术从光流场中提取分割运动肌肉。在公开数据集MPI Sintel、KITTI2012、KITTI2015、Flying Chairs和CityScapes(共6 000个基本模型的样本和3 600个多帧模型的样本)进行了基本测试、跨数据集泛化测试,在600张超声图像上进行了迁移测试。结果表明,该光流算法在不借助数据集光流标签的前提下可以有效地提取稠密光流场信息,平均端点误差为5.80,模型参数量降低到2.35 M,且具有较好的跨数据集泛化能力。在迁移测试中,与人工分割结果相比中心点偏移差值均值限制在1 mm以内,交并比评价指标均值高于0.9。实验结果表明该方法可以有效分割超声图像下的运动肌肉。

	服务

	把本文推荐给朋友
	加入我的书架
	加入引用管理器
	E-mail Alert
	RSS
	作者相关文章
	李扬
	李金
	栾宽

关键词 ：靶肌分割, 稠密光流, 卷积神经网络, 无监督学习

Abstract：Accurate image segmentation is the foundation of image guided treatment for spastic muscles after stroke. We proposed an unsupervised deep learning optical flow algorithm, aiming to segment the moving target muscle. Due to lack of labels of ultrasound image dataset of motor muscles not like the previous static ultrasound image datasets, an unsupervised optical flow estimation framework was constructed. The optical flow was learned from the unlabeled image sequence through augemented self-monitoring. At the same time, in order to avoid the influence of view synthesis targets on accuracy of augmented data, another forward propagation was added to the converted image to distort the basic learning framework, in which the conversion prediction from the original image is monitored, and finally, Gaussian filter, speed threshold filtering, border detection filtering and convex hull segmentation were employed to extract moving muscles from the optical flow field. The proposed method was evaluated on MPI Sintel, KITTI2012, KITTI2015, Flying Chairs and CityScapes (totally including 6 000 basic samples and 3 600 multi frame model samples) for its basic performance and cross dataset generalization performance, and muscle ultrsound image dataset containing 600 samples for its transfer performance was tested. The results demonstrated that the proposed method effectively extracted the dense optical flow field information without the use of optical flow labels in the dataset. The average endpoint error reached 5.80, the parameter quantity was reduced to 2.35 M, and good cross dataset generalization ability was obsersed. In the transfer test, compared with the manual segmentation results, the mean of center point differences was less than 1 mm, and the mean of intersection over unions was higher than 0.9. The proposedmethod can effectively segment the moving musclesof ultrasound images.

Key words： target muscle segmentation dense optical flow convolutional neural network unsupervised learning

收稿日期: 2022-03-18

PACS:

R318

基金资助:国家自然科学基金(81301297,61773134)

通讯作者: ^*E-mail:luankuan@hrbeu.edu.cn

作者简介: ^#中国生物医学工程学会会员

引用本文:

李扬, 李金, 栾宽. 基于无监督深度学习光流法的超声图像运动肌肉分割[J]. 中国生物医学工程学报, 2025, 44(4): 435-446.
Li Yang, Li Jin, Luan Kuan. Muscle Ultrasound Image Segmentation Based on Unsupervised Deep Learning Optical Flow Method. Chinese Journal of Biomedical Engineering, 2025, 44(4): 435-446.

链接本文:

http://cjbme.csbme.org/CN/10.3969/j.issn.0258-8021.2025.04.006 或 http://cjbme.csbme.org/CN/Y2025/V44/I4/435

[1] 何思佳,杨信才. 脑卒中偏瘫患者肌肉痉挛治疗的研究进展[J]. 医学研究与教育, 2021, 38(2):26-31.
[2] 张健. 痉挛的药物和手术治疗[J]. 继续医学教育, 2013, 27(8): 27-29.
[3] Fischer P, Dosovitskiy A, Ilg E, et al. FlowNet: Learning Optical Flow with Convolutional Networks [C]//Proceedings of the 2015 IEEE International Conference on Computer Vision. Santiago: IEEE, 2015:2758-2766.
[4] Ilg E, Mayer N, Saikia T, et al. FlowNet 2.0: evolution of optical flow estimation with deep networks [C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 1647-1655.
[5] Sun Deqing, Yang Xiaodong, Liu Ming-Yu, et al. PWC-Net: CNNs for optical flow using pyramid, warping, and cost volume [C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 8934-8943.
[6] Hur J, Roth S. Iterative residual refinement for joint optical flow and occlusion estimation [C]//Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 5747-5756.
[7] Mayer N, Ilg E, Hausser P, et al. A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 4040-4048.
[8] Yu J, Harley A W, Derpanis K G. Back to basics: Unsupervised learning of optical flow via brightness constancy and motion smoothness [C]//Proceedings of the 2016 European Conference on Computer Vision. Amsterdam: Springer, 2016:3-10.
[9] Wang Yang, Yang Yi, Yang Zhenheng, et al. Occlusion aware unsupervised learning of optical flow [C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 4884-4893.
[10] Meister S, Hur J, Roth S. Unflow: unsupervised learning of optical flow with a bidirectional census loss [C]//Proceedings of the 2018 AAAI Conference on Artificial Intelligence. New Orleans: AAAI, 2018: 7251-7259.
[11] Janai J, Güney F, Ranjan A, et al. Unsupervised learning of multi-frame optical flow with occlusions[C]//Proceedings of the 2018 European Conference on Computer Vision. Cham:Springer, 2018: 713-731.
[12] Guan Shuosen, Li Haoxin, Zheng Wei-Shi. Unsupervised learning for optical flow estimation using pyramid convolution lstm [C]//Proceedings of the 2019 IEEE International Conference on Multimedia and Expo. Shanghai: IEEE, 2019: 181-186.
[13] Zhong Yiran, Ji Pan, Wang Jianyuan, et al. Unsupervised deep epipolar flow for stationary or dynamic scenes [C]//Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 12087-12096.
[14] Wang Yang, Wang Peng, Yang Zhenheng, et al. Unos: unified unsupervised optical flow and stereo-depth estimation by watching videos[C]//Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 8063-8073.
[15] Liu Liang, Zhai Guangyao, Ye Wenlong, et al. Unsupervised learning of scene flow estimation fusing with local rigidity [C]//Proceedings of the 2019 International Joint Conference on Artificial Intelligence. Macao: AAAI, 2019: 876-882.
[16] Zou Yuliang, Luo Zelun, Huang Jia-Bin. Df-net: Unsupervised joint learning of depth and flow using cross-task consistency [C]//Proceedings of the 2018 European Conference on Computer Vision. Cham:Springer, 2018: 38-55.
[17] Liu Pengpeng, King I, Lyu MR, et al. DDFlow: learning optical flow with unlabeled data distillation [C]//Proceedings of the AAAI Conference on Artificial Intelligence. Honolulu: AAAI, 2019: 8770-8777.
[18] Liu Pengpeng, Lv M, King I, et al. Selflow: Self-supervised learning of optical flow [C]//Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 4566-4575.
[19] Butler DJ, Wulff J, Stanley GB, et al. A naturalistic open source movie for optical flow evaluation[C]//Proceedings of the 2012 European Conference on Computer Vision. Berlin:Springer, 2012: 611-625.
[20] Geiger A, Lenz P, Urtasun R. Are we ready for autonomous driving? The kitti vision benchmark suite[C]//Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition. Providence: IEEE, 2012: 3354-3361.
[21] Menze M, Geiger A. Object scene flow for autonomous vehicles[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston: IEEE, 2015: 3061-3070.
[22] Cordts M, Omran M, Ramos S, et al. The cityscapes dataset for semantic urban scene understanding[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 3213-3223.
[23] Rohit G, Alaaeldin EN, Mannat S, et al. OmniMAE: Single Model Masked Pretraining on Images and Videos[C]//Proceedings of the 2021 Neural Information Processing System Workshop. New Orleans: Springer, 2022: 1-19.