Abstract：Statistically counting the number of mitotic cells in histological images of breast tumor tissue is an important diagnostic basis for the grading and prognosis of breast cancer. Currently, the counting tasks are performed manually by pathologists, which is a time-consuming and laborious task. To address this challenge, this paper proposed a method for mitotic detection in breast cancer pathology images from local to regional stratification. The framework as a whole consisted of two stages. The first stage was a cell localization network, which was responsible for screening and locating candidate mitotic cell blocks from whole section images while introducing a deep supervision mechanism with decoupled detection heads to enhance performance. The second stage was the mitotic cell validation network, which was responsible for further refining the classification of a large number of candidate cell image blocks by using a contextual fusion network based on a graph-attention mechanism to modulate the original response of local central blocks by integrating a large range of regional features to obtain more accurate classification results. We achieved F-Scores of 0.676, 0.809, and 0.797 on the ICPR MITOSIS 2014, ICPR MITOSIS 2012, and TUPAC16 datasets, respectively, using 960, 35, and 649 HPF as training sets, and 240, 15, and 7 HPF as test sets, respectively, where the recall rates all achieved optimal results of 0.878, 0.858 and 0.875, respectively. The results indicated that the proposed automatic detection method efficiently detected the cancer cells in the pathological sections, showing significant clinical application value.
蔡玉, 唐奇伶, 刘子仪. 从局部到区域分层的乳腺病理图像有丝分裂检测[J]. 中国生物医学工程学报, 2023, 42(6): 687-697.
Cai Yu, Tang Qiling, Liu Ziyi. Mitotic Detection in Breast Histopathology Images Using Local and Regional Hierarchical Information. Chinese Journal of Biomedical Engineering, 2023, 42(6): 687-697.
[1] Siegel RL, Miller KD, Fuchs HE, et al. Cancer Statistics, 2022 [J]. CA: Cancer J Clin, 2022, 72(1): 7-33. [2] Frierson Jr HF, Wolber RA, Berean KW, et al. Interobserver reproducibility of the Nottingham modification of the Bloom and Richardson histologic grading scheme for infiltrating ductal carcinoma [J]. American Journal of Clinical Pathology, 1995, 103(2): 195-198. [3] 王永军,黄芳琳,黄珊,等. 基于融合多网络深层卷积特征和稀疏双关系正则化方法的乳腺癌图像分类研究 [J]. 中国生物医学工程学报, 2020, 39(5): 532-540. [4] 周超,徐军,罗波. 基于深度卷积网络和结合策略的乳腺组织病理图像细胞核异型性自动评分 [J]. 中国生物医学工程学报, 2017, 36(3): 276-283. [5] Cireşan DC, Giusti A, Gambardella LM, et al. Mitosis detection in breast cancer histology images with deep neural networks [C]// International Conference on Medical Image Computing and Computer-Assisted Intervention. Berlin: Springer, 2013: 411-418. [6] Wang Haibo, Roa AC, Basavanhally AN, et al. Mitosis detection in breast cancer pathology images by combining handcrafted and convolutional neural network features [J]. Journal of Medical Imaging, 2014, 1(3): 034003. [7] Li Chao, Wang Xinggang, Liu Wenyu, et al. Weakly supervised mitosis detection in breast histopathology images using concentric loss [J]. Medical Image Analysis, 2019, 53: 165-178. [8] Chen Hao, Wang Xi, Heng PA. Automated mitosis detection with deep regression networks [C]//2016 IEEE 13th International Symposium on Biomedical Imaging. Prague: IEEE, 2016: 1204-1207. [9] Girshick R. Fast R-CNN [C]//Proceedings of the IEEE International Conference on Computer Vision. Santiago: IEEE, 2015: 1440-1448. [10] Ren S, He Kaiming, Girshick R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017, 39(6): 1137-1149. [11] Mahmood T, Arsalan M, Owais M, et al. Artificial intelligence-based mitosis detection in breast cancer histopathology images using faster R-CNN and deep CNNs [J]. Journal of Clinical Medicine, 2020, 9(3): 749-774. [12] Cai De, Sun Xianhe, Zhou Niyun, et al. Efficient mitosis detection in breast cancer histology images by RCNN [C]//2019 IEEE 16th International Symposium on Biomedical Imaging. Venice: IEEE, 2019: 919-922. [13] Li Chao, Wang Xinggang, Liu Wenyu, et al. DeepMitosis: Mitosis detection via deep detection, verification and segmentation networks [J]. Medical Image Analysis, 2018, 45: 121-133. [14] Das DK, Dutta PK. Efficient automated detection of mitotic cells from breast histological images using deep convolution neutral network with wavelet decomposed patches [J]. Computers in Biology and Medicine, 2019, 104: 29-42. [15] Sohail A, Khan A, Nisar H, et al. Mitotic nuclei analysis in breast cancer histopathology images using deep ensemble classifier [J]. Medical Image Analysis, 2021, 72: 102-121. [16] Lee CY, Xie Saining, Gallagher P, et al. Deeply-supervised nets [C]//Artificial Intelligence and Statistics. New York: IEEE, 2015: 562-570. [17] Liu Yun, Cheng Mingming, Hu Xiaowei, et al. Richer convolutional features for edge detection [C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 3000-3009. [18] Veličković P, Cucurull G, Casanova A, et al. Graph attention networks [DB/OL]. https://arxiv.org/abs/1710.10903, 2018-02-04/2023-08-18. [19] Veta M, Heng YJ, Stathonikos N, et al. Predicting breast tumor proliferation from whole-slide images: the TUPAC16 challenge [J]. Medical Image Analysis, 2019, 54: 111-121. [20] Ludovic R, Daniel R, Nicolas L, et al. Mitosis detection in breast cancer histological images An ICPR 2012 contest [J]. Journal of Pathology Informatics, 2013, 4(1): 8. [21] Reinhard E, Adhikhmin M, Gooch B, et al. Color transfer between images [J]. IEEE Computer Graphics and Applications, 2001, 21(5): 34-41. [22] He Kaiming, Zhang Xianyu, Ren Shaoqing, et al. Deep residual learning for image recognition [C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778. [23] Zhang Si, Tong Hanghang, Xu Jiejun, et al. Graph convolutional networks: a comprehensive review [J]. Computational Social Networks, 2019, 6(1): 1-23. [24] Deng Jia, Dong Wei, Socher R, et al. Imagenet: A large-scale hierarchical image database [C]//2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami Beach: IEEE, 2009: 248-255. [25] Dodballapur V, Song Yang, Huang Heng, et al. Mask-driven mitosis detection in histopathology images [C]//2019 IEEE 16th International Symposium on Biomedical Imaging. Venice: IEEE, 2019: 1855-1859. [26] Chen Hao, Dou Qi, Wang Xi, et al. Mitosis detection in breast cancer histology images via deep cascaded networks [C]//Thirtieth AAAI Conference on Artificial Intelligence. Arizona: AAAI Press, 2016: 1160-1166. [27] Han Jiangxiao, Wang Xinggang, Liu Wenyu. Contextual prior constrained deep networks for mitosis detection with point annotations [J]. IEEE Access, 2021, 9: 71954-71967. [28] Li Yonghui, Xue Yao, Li Liangfu, et al. Domain adaptive box-supervised instance segmentation network for mitosis detection[J]. IEEE Transactions on Medical Imaging, 2022, 41(9): 2469-2485. [29] Zerhouni E, Lányi D, Viana M, et al. Wide residual networks for mitosis detection [C]//2017 IEEE 14th International Symposium on Biomedical Imaging. Melbourne: IEEE, 2017: 924-928. [30] Kugler M, Goto Y, Kawamura N, et al. Accurate 3D reconstruction of a whole pancreatic cancer tumor from pathology images with different stains[C]//Computational Pathology and Ophthalmic Medical Image Analysis: First International Workshop, COMPAY 2018. Granada: Springer International Publishing, 2018: 35-43. [31] Xu Jun, Xiang Lei, Liu Qingshan, et al. Stacked sparse autoencoder (SSAE) for nuclei detection on breast cancer histopathology images [J]. IEEE Transactions on Medical Imaging, 2016, 35(1): 119-130. [32] Beck AH, Sangoi AR, Leung S, et al. Systematic analysis of breast cancer morphology uncovers stromal features associated with survival [J]. Science Translational Medicine, 2011, 3(108): 108-113. [33] Komura D, Ishikawa S. Machine learning methods for histopathological image analysis [J]. Computational and Structural Biotechnology Journal, 2018, 16: 34-42.
