基于通道注意力机制和多尺度旋转卷积的Mamba双分支医学图像融合模型

doi:10.3969/j.issn.0258-8021.2026.02.002

Abstract
Figure/Table
References
Related Citation (15)

Download: PDF (8908 KB) HTML (1 KB)
Export: BibTeX | EndNote (RIS)

Abstract To address the high computational complexity of Transformers, a Mamba dual-branch fusion network based on channel attention mechanism and multi-scale rotation convolution (rotation-enhanced Mamba fusion model, ReMFM) was proposed in this work. First, the channel attention mechanism was ued to capture the inter-channel information correlation of the images to be fused. Next, a rotation convolution module was designed to efficiently extract cross-modal local structural features in both direction and scale dimensions. Finally, an improvement was made to Mamba by designing an attention state space module, which introduced non-causal modeling and global perception capability with single-scan processing. These actions significantly reduced computational complexity and redundancy while ensuring expressiveness. The data used in this experiment were obtained from the Harvard Brain Atlas Database, comprising 357 paired MRI and SPECT images, with 333 pairs allocated for training and 24 pairs for testing. Data augmentation was performed using an overlapping cropping strategy, and all training images were standardized to a uniform size of 120 × 120 pixels. Experimental results showed that ReMFM achieved 0.7438, 0.7457, 0.9767, 0.6884, and 5.0226 in gradient, image feature, Yann measure, visual information fidelity, and mutual information, respectively, with improvements of 2.52%, 15.92%, 2.41%, 14.64%, and 14.29% over seven mainstream Transformer-based models. The proposed model effectively highlights the structural information of the lesion regions while preserving edge textures, producing high-quality fused images.

Key words： medical image fusion medical images attention mechanisms Mamba

Received: 11 November 2025

PACS:

R318

Corresponding Authors: ^*E-mail:kwwking@163.com

	Service

	E-mail this article
	Add to my bookshelf
	Add to citation manager
	E-mail Alert
	RSS
	Articles by authors
	Kong Weiwei
	Han Yinbo
	Lei Yang
	Wang Yuchen
	Zhou Haochen

Cite this article:

Kong Weiwei,Han Yinbo,Lei Yang, et al. Mamba Dual-Branch Medical Image Fusion Model Based on Channel Attention Mechanism and Multi-Scale Rotational Convolution[J]. Chinese Journal of Biomedical Engineering, 2026, 45(2): 141-153.

URL:

http://cjbme.csbme.org/EN/10.3969/j.issn.0258-8021.2026.02.002 OR http://cjbme.csbme.org/EN/Y2026/V45/I2/141

[1] 陈伟, 孙康康, 李奇轩, 等. 多模态医学图像融合方法的研究进展[J]. 中国辐射卫生, 2023, 32(5): 580-585.
[2] 史韶杰, 韩建宁, 李大威, 等. 基于多尺度曝光融合的医学 CT 图像增强方法[J]. 电子测量技术, 2024, 45(6): 106-111.
[3] Ma J, Ma Y, Li C. Infrared and visible image fusion methods and applications: A survey[J]. Information Fusion, 2019, 45(1): 153-178.
[4] James AP, Dasarathy BV. Medical image fusion: A survey of the state of the art[J]. Information Fusion, 2014, 19(9): 4-19.
[5] Li H, Wu XJ. DenseFuse: A fusion approach to infrared and visible images[J]. IEEE Transactions on Image Processing, 2018, 28(5): 2614-2623.
[6] Xu H, Ma J, Jiang J, et al. U2Fusion: A unified unsupervised image fusion network[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 44(1): 502-518.
[7] Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[J]. Advances in Neural Information Processing Systems, 2017, 30.
[8] Fu Y, Xu TY, Wu XJ, et al. PPT fusion: Pyramid patch transformerfor a case study in image fusion[J/OL]. https://arxiv.org/abs/2107.13967,2022-08-03/2025-11-03.
[9] Wang Z, Chen Y, Shao W, et al. SwinFuse: A residual swin transformer fusion network for infrared and visible images[J]. IEEE Transactions on Instrumentation and Measurement, 2022, 71(7): 5016412.
[10] Ma J, Tang L, Fan F, et al. SwinFusion: Cross-domain long-range learning for general image fusion via swin transformer[J]. IEEE/CAA Journal of Automatica Sinica, 2022, 9(7): 1200-1217.
[11] Qu L, Liu S, Wang M, et al. Transmef: a transformer-based multi-exposure image fusion framework using self-supervised multi-task learning[C]//Proceedings of the AAAI Conference on Artificial Intelligence. Vancouver: AAAI Press, 2022, 36(2): 2126-2134.
[12] Zhang Y, Liu H, Hu Q. Transfuse: fusing transformers and cnns for medical image segmentation[C]//Medical Image Computing and Computer Assisted Intervention-MICCAI 2021: 24th International Conference.Strasbourg: Springer International Publishing, 2021: 14-24.
[13] Tang W, He F, Liu Y. YDTR: infrared and visible image fusion via Y-shape dynamic transformer[J]. IEEE Transactions on Multimedia, 2022, 25(7): 5413-5428.
[14] Tang W, He F, Liu Y. ITFuse: an interactive transformer for infrared and visible image fusion[J]. Pattern Recognition, 2024, 156(12): 110822.
[15] Jian L, Xiong S, Yan H, et al. Rethinking cross-attention for infrared and visible image fusion[J/OL]. https://arxiv.org/abs/2401.11675, 2025-06-18, 2025-11-03.
[16] Yang X, Huo H, Li C, et al. Semantic perceptive infrared and visible image fusion transformer[J]. Pattern Recognition, 2024, 149(5): 110223.
[17] Zhao Y, Zheng Q, Zhu P, et al. TUFusion: a transformer-based universal fusion algorithm for multimodal images[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2023, 34(3): 1712-1725.
[18] Fang L, Hou M, Huang B, et al. DCAFusion: a novel general image fusion framework based on reference image reconstruction and dual-cross attention mechanism[J]. Information Sciences, 2025, 698(4): 121772.
[19] Tang W, He F, Liu Y, et al. MATR: multimodal medical image fusion via multiscale adaptive transformer[J]. IEEE Transactions on Image Processing, 2022, 31(7): 5134-5149.
[20] Gu A, Dao T. Mamba: Linear-time sequence modeling with selective state spaces[C]//Proceedings of the First Conference on Language Modeling. Philadelphia: OpenReview.net, 2024: 1-32.
[21] Liu Y, Tian Y, Zhao Y, et al. Vmamba: Visual state space model[J]. Advances in Neural Information Processing Systems, 2024, 37: 103031-103063.
[22] Yin M, Liu X, Liu Y, et al. Medical image fusion with parameter-adaptive pulse coupled neural network in nonsubsampled shearlet transform domain[J]. IEEE Transactions on Instrumentation and Measurement, 2018, 68(1): 49-64.
[23] Guo H, Guo Y, Zha Y, et al. Mambairv2: Attentive state space restoration[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2025: 28124-28133.
[24] Xu H, Ma J. EMFusion: An unsupervised enhanced medical image fusion network[J]. Information Fusion, 2021, 76(12): 177-186.
[25] Kingma D P. Adam: A method for stochastic optimization[J/OL]. https://arxiv.org/abs/1412.6980,2017-01-30, 2025-11-03.
[26] Tang W, He F. FATFusion: a functional-anatomical transformer for medical image fusion[J]. Information Processing & Management, 2024, 61(4): 103687.
[27] Xie X, Zhang X, Tang X, et al. MACTFusion: lightweight cross transformer for adaptive multimodal medical image fusion[J]. IEEE Journal of Biomedical and Health Informatics, 2025, 29(5): 3317-3328.
[28] Zhao Z, Deng L, Bai H, et al. Image fusion via vision-language model[J/OL]. https://arxiv.org/abs/2402.02235,2024-07-10, 2025-11-03.
[29] Hossny M, Nahavandi S, Creighton D. Comments on ‘Information measure for performance of image fusion’[J]. Electronics Letters, 2008, 44(18): 1066-1067.
[30] Xydeas CS, Petrovic V. Objective image fusion performance measure[J]. Electronics letters, 2000, 36(4): 308-309.
[31] Zhao J, Laganiere R, Liu Z. Performance assessment of combinative pixel-level image fusion based on an absolute feature measurement[J]. Int J Innov Comput Inf Control, 2007, 3(6): 1433-1447.
[32] Yang C, Zhang JQ, Wang XR, et al. A novel similarity based quality metric for image fusion[J]. Information Fusion, 2008, 9(2): 156-160.
[33] Chen Y, Blum RS. A new automated quality assessment algorithm for image fusion[J]. Image and Vision Computing, 2009, 27(10): 1421-1432.
[34] Sheikh HR, Bovik AC. Image information and visual quality[J]. IEEE Transactions on Image Processing, 2006, 15(2): 430-444.
[35] Peng H, Long F, Ding C. Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2005, 27(8): 1226-1238.
[36] Bekkers E J, Lafarge M W, Veta M, et al. Roto-translation covariant convolutional networks for medical image analysis[C]//Medical Image Computing and Computer Assisted Intervention-MICCAI 2018: 21st International Conference, Granada: Springer International Publishing, 2018: 440-448.
[37] Hu Q, Peng Y. Infrared and visible image fusion using a state-space adversarial model with cross-Modal dependency learning[J]. Mathematics, 2025, 13(15): 2333.
[38] Hasan BT, Al-Sabaawi AMA, Al-Araji ZJ. Lightweight deep learning for IoT-based medical diagnosis: a survey[J]. Mesopotamian Journal of Artificial Intelligence in Healthcare, 2025, 2025: 244-259.