Abstract：Spatial and temporal features obtained from dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) are both used for the prediction of breast tumors. Previous studies mainly focused on the shallow spatiotemporal features, and the effectiveness of the deep spatiotemporal features is still unclear. To this end, a multiscale residual spatiotemporal model (MRSM) based on ResNet (2+1)D was proposed for breast tumor prediction. This study included 232 patients from Taizhou Central Hospital, with 85 benign cases and 147 malignant cases. The DCE-MRI data consisted of one precontrast image and eight postcontrast images.Compared with the original ResNet (2+1)D, MRSM method added a multiscale module and residual module. Specifically, the multi-scale module employed two independent dual-path spatial convolutions and dual-path temporal convolutions to effectively capture the spatio-temporal features of the breast tumor. The performance of the MRSM method was evaluated using the area under the receiver operating characteristic (ROC) curve (AUC). Compared to the ResNet (2+1)D method (AUC=0.974 0) or existing methods (The best AUC=0.960 8), this method achieved a higher AUC of 0.987 0.In conclusion, the MRSM method is an effective approach for accurate prediction of breast tumors.
张亮亮, 郑航, 刘家伟, 王朕朕, 毕科健. 基于多尺度残差时空模型的乳腺肿瘤预测[J]. 中国生物医学工程学报, 2026, 45(1): 38-46.
Zhang Liangliang, Zheng Hang, Liu Jiawei, Wang Zhenzhen, Bi Kejian. Breast Tumor Prediction Based on Multiscale Residual Spatiotemporal Model. Chinese Journal of Biomedical Engineering, 2026, 45(1): 38-46.
[1] Siegel RL, Miller KD, Fuchs HE, et al. Cancer statistics[J]. CA Cancer J Clin, 2022, 72(1): 7-33. [2] Fan Ming, Xia Pingping, Clarke R, et al. Radiogenomic signatures reveal multiscale intratumour heterogeneity associated with biological functions and survival in breast cancer[J]. Nature Communications, 2020, 11(1): 4861. [3] Zhang Liangliang, Fan Ming, Li Lihua. Efficient estimation of pharmacokinetic parameters from breast dynamic contrast-enhanced MRI based on a convolutional neural network for predicting molecular subtypes[J]. Physics in Medicine & Biology, 2023, 68(24): 245001. [4] Lambin P, Leijenaar RT, Deist TM, et al. Radiomics: the bridge between medical imaging and personalized medicine[J]. Nature reviews Clinical oncology, 2017, 14(12): 749-762. [5] Fan Ming, Xia Pingping, Liu Bin, et al. Tumour heterogeneity revealed by unsupervised decomposition of dynamic contrast-enhanced magnetic resonance imaging is associated with underlying gene expression patterns and poor survival in breast cancer patients[J]. Breast Cancer Research, 2019, 21: 1-16. [6] Fan Ming, Yuan Chengcheng, Huang Guangyao, et al. A framework for deep multitask learning with multiparametric magnetic resonance imaging for the joint prediction of histological characteristics in breast cancer[J]. IEEE Journal of Biomedical Health Informatics, 2022, 26(8): 3884-3895. [7] Zhou Xinxiang, Zhang Lan, Cui Quanxiang, et al. A Channel-dimensional feature-reconstructed deep learning model for predicting breast cancer molecular subtypes on overall b-value diffusion-weighted MRI[J]. Journal of Magnetic Resonance Imaging, 2024, 59(4): 1425-1435. [8] Cong Chao, Li Xiaoguang, Zhang Chunlai, et al. MRI-based breast cancer classification and localization by multiparametric feature extraction and combination using deep learning[J]. Journal of Magnetic Resonance Imaging, 2024, 59(1): 148-161. [9] Hasan AM, Al-Waely NKN, Aljobouri HK, et al. Molecular subtypes classification of breast cancer in DCE-MRI using deep features[J]. Expert Systems with Applications, 2024, 236: 121371. [10] Banaie M, Soltanian-Zadeh H, Saligheh-Rad HR, et al. Spatiotemporal features of DCE-MRI for breast cancer diagnosis[J]. Computer Methods and Programs in Biomedicine, 2018, 155: 153-164. [11] Caballo M, Sanderink WBG, Han Luyi, et al. Four-dimensional machine learning radiomics for the pretreatment assessment of breast cancer pathologic complete response to neoadjuvant chemotherapy in dynamic contrast-enhanced MRI[J]. Journal of Magnetic Resonance Imaging, 2023, 57(1): 97-110. [12] Chatterjee S, Nizamani FA, Nürnberger A, et al. Classification of brain tumours in MR images using deep spatiospatial models[J]. Scientific Reports, 2022, 12(1): 1505. [13] Tran D, Wang Heng, Torresani L, et al. A closer look at spatiotemporal convolutions for action recognition[C]// Proceedings of the IEEE conference on Computer Vision and Pattern Recognition. Salt Lake City:IEEE, 2018: 6450-6459. [14] Qiu Zhaofan, Yao Ting, Mei Tao. Learning spatio-temporal representation with pseudo-3D residual networks[C]// Proceedings of the IEEE International Conference on Computer Vision.Venice: IEEE, 2017: 5533-5541. [15] Zhao Xiaoming, Liao Yuehui, Xie Jiahao, et al. BreastDM: A DCE-MRI dataset for breast tumor image segmentation and classification[J]. Computers in Biology and Medicine, 2023, 164: 107255. [16] Hara K, Kataoka H, Satoh Y. Can spatiotemporal 3D cnns retrace the history of 2d cnns and imagenet?[C]//Proceedings of the IEEE conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE. 2018: 6546-6555. [17] Kopuklu O, Kose N, Gunduz A, et al. Resource efficient 3d convolutional neural networks[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops. Venice: IEEE, 2019: 1910-1919. [18] Huang Gao, Liu Zhuang, Van Der Maaten L, et al. Densely connected convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu:IEEE, 2017: 4700-4708. [19] 谈咏东, 王永雄, 陈姝意, 等. (2+1) D 多时空信息融合模型及在行为识别的应用[J]. 信息与控制, 2019, 48(6): 715-722. [20] 庄建军, 吴晓慧, 景生华, 等. 多尺度特征融合的改进残差网络乳腺癌病理图像分类[J]. 中国生物医学工程学报, 2024, 43(4): 419-428. [21] 王勇, 靳伟昭, 冯伟, 等. 一种改进 R (2+1) D 网络的暴力行为检测方法[J]. 西安电子科技大学学报, 2022, 49(2): 155-163. [22] Delong ER, Delong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach[J]. Biometrics, 1988, 44(3): 837-845. [23] Wang Xianyang, Lv Linlin, Tang Qingfeng, et al. A feature fusion method based on radiomic features and revised deep features for improving tumor prediction in ultrasound images[J]. Computers in Biology Medicine, 2025, 185: 109605. [24] Atrey K, Singh BK, Bodhey NK. Multimodal classification of breast cancer using feature level fusion of mammogram and ultrasound images in machine learning paradigm[J]. Multimedia Tools Applications, 2024, 83(7): 21347-21368. [25] Jiang Shengqin, Zhang Haokui, Qi Yuankai, et al. Spatial-temporal interleaved network for efficient action recognition[J]. IEEE Transactions on Industrial Informatics, 2024, 21(1): 178-187. [26] Xu Cheng, Yi Ke, Jiang Nan, et al. MDFF-Net: A multi-dimensional feature fusion network for breast histopathology image classification[J]. Computers in Biology Medicine, 2023, 165: 107385. [27] Nguyen S, Polat D, Karbasi P, et al. Preoperative prediction of lymph node metastasis from clinical DCE MRI of the primary breast tumor using a 4D CNN[C]//Medical Image Computing and Computer Assisted Intervention-MICCAI. Lima:MICCAI, 2020: 326-334. [28] Ulas C, Tetteh G, Thrippleton MJ, et al. Direct estimation of pharmacokinetic parameters from DCE-MRI using deep CNN with forward physical model loss[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention. Granada: MICCAI, 2018: 39-47.
