A Skin Cancer Detection Framework Based on Double-Branch Attention Neural Networks
Wang Yufeng1*, Cheng Haoyuan1, Wan Chengbei1, Zhang Bo2, Shi Aiju2
1(School of Telecommunications and Information Engineering, Nanjing University of Posts and Telecommunications, Nanjing 210003, China) 2(College of Science, Nanjing University of Posts and Telecommunications, Nanjing 210046, China)
Abstract：Skin cancer is a major cancerand has increased rapidly in the past decades. Early detection can significantly increase the cure rate. Recently, deep learning models, especially various convolutional Neural Networks using dermatoscope images (i.e., dermoscopy) have been widely adopted to classify skin lesions. Different from traditional image classification, several challenges in detecting and classifying skin cancers still exist, including imbalanced training data in each skin cancer category, small visual differences between categories, and small area of skin lesion. To solve these challenges, this paper proposed a skin cancer classification framework based on double-branch attention convolutional neural networks (DACNN). First, in data pre-processing, the whole dataset was divided into finer-grained categories according to the natural sub-classes in each category to alleviate the imbalanced data. Next, from the viewpoint of neural network structure, attention residual learning (ARL) modules were used as basic blocks in upper-branch, which was able to effectively extract the features of potential sick area, then thelesion location network (LLN) was designed to localize, cut out and zoom-in the sick sub-area, followed by being sent to down-branch with the same neural structure as the upper-branch, for extracting the locally detailed features. Then, the inferred features from both branches were integrated for effective detection and classification. Moreover, to further alleviate the impact of imbalanced categorical data, weighted loss function was utilized in the model training. The proposed DACNN model was implemented in the real dataset consisting of 10015 dermatoscope images and compared with several typical deep learning based skin lesion detection methods. Experimental results showed that the performance metrics of sensitivity, accuracy and F1_score reached 0.922, 0.942 and 0.933, respectively. Compared with recurrent attention convolutional neural network (RACNN) detection methods, these three metrics were improved by 3.48%, 2.95% and 3.44% respectively. In summary, our work significantly improved the accuracy of dermoscopy based skin cancer detection through appropriate division of dermatoscope image classes, used the double-branch attention neural networks to firstly localize and enlarge the features of potential sick area, and then further extracted the locally detailed features, which solved the intrinsic issues of dermatoscope images, including imbalanced samples in each skin cancer category, vague visual differences between categories, and small areaof skin lesion.
王玉峰, 成昊沅, 万承北, 张博, 石爱菊. 一种基于双分支注意力神经网络的皮肤癌检测框架[J]. 中国生物医学工程学报, 2024, 43(2): 153-161.
Wang Yufeng, Cheng Haoyuan, Wan Chengbei, Zhang Bo, Shi Aiju. A Skin Cancer Detection Framework Based on Double-Branch Attention Neural Networks. Chinese Journal of Biomedical Engineering, 2024, 43(2): 153-161.
[1] Apalla Z, Nashan D, Weller RB, Castellsague X. Skin cancer: epidemiology, disease burden, pathophysiology, diagnosis, and therapeutic approaches [J]. Dermatol. Therapy, 2017, 7(1): 5-19. [2] Kittler H, Pehamberger H, Wolff K, Binder M. Diagnostic accuracy of dermoscopy [J]. Lancet Oncol, 2002, 3(3): 159-165. [3] Vestergaard ME, Macaskill P, Holt PE, et al. Dermoscopy compared with naked eye examination for the diagnosis of primary melanoma: a meta analysis of studies performed in a clinical setting [J]. Brit J Dermatol, 2008, 159(3): 669-676. [4] Argenziano G, Fabbrocini G, Carli P, et al. Epiluminescence microscopy for the diagnosis of doubtful melanocytic skin lesions: comparison of the ABCD rule of dermatoscopy and a new 7-point checklist based on pattern analysis [J]. Arch Dermatol, 1998, 134(12): 1563-1570. [5] Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks [C] //Proceedings of the 26th Annual Conference on Neural Information Processing Systems (NIPS). Lake Tahoe: ACM, 2012: 1106-1114. [6] He Kaiming, Zhang Xiangyu, Ren Shaoqing Ren, et al. Deep residual learning for image recognition [C] //Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas: IEEE, 2016: 770-778. [7] Huang Gao, Liu Zhuang, Van Der Maaten L, et al.. Densely connected convolutional networks [C] //IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu: IEEE, 2017: 4700-4708. [8] Howard AG, Zhu Menglong, Chen Bo, et al. Mobilenets: efficient convolutional neural networks for mobile vision applications [EB/OL]. https://arxiv.org/abs/1704.04861, 2017:4-17/ 2021:11-27. [9] Esteva A, Kuprel B, Novoa RA, et al. Dermatologist-level classification of skin cancer with deep neural networks [J]. Nature, 2017, 542(7639): 115-118. [10] Zhang Jianpeng, Xie Yutong, Xia Yong, et al. Attention residual learning for skin lesion classification [J]. IEEE Trans. on Medical Imaging, 2019, 38(9): 2092-2103. [11] Gessert N, Sentker T, Madesta F, et al. Skin lesion classification using CNNs with patch-based attention and diagnosis-guided loss weighting [J]. IEEE Trans. on Biomedical Engineering, 2020, 67(2): 495-503. [12] Fu Jianlong, Zhang Heliang, Mei Tao. Look closer to see better: Recurrent attention convolutional neural network for fine-grained image recognition [C] //Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu: IEEE, 2017: 4476-4484. [13] Pillay V, Hirasen D, Viriri S, et al. Melanoma skin cancer classification using transfer learning [C] // Proceedings of the 12th International Conference on Computational Collective Intelligence. Berlin: Springer, 2020: 287-297. [14] Wang Fei, Jiang Mengqing, Qian Chen, et al. Residual attention network for image classification [C] //Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu: IEEE, 2017: 6450-6458. [15] Hu Jie, Shen Li, Sun Gang. Squeeze-and-excitation networks [C] //Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Salt Lake City: IEEE, 2018: 7132-7141. [16] Tschandl P, Rosendahl C, Kittler H. The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. [J]. Sci Data, 2018, 5(1): 180161. [17] Mahbod A, Schaefer G, Wang Chunliang, et al. Transfer learning using a multi-scale and multi-network ensemble for skin lesion classification [J]. Computer Methods and Programs in Biomedicine, 2020, 193: 105475. [18] Song Lei, Lin Jianzhe, Wang Z Jane, et al. An end-to-end multi-task deep learning framework for skin lesion analysis [J]. IEEE Journal of Biomedical and Health Informatics, 2020, 24(10): 2912-2921. [19] Yang Jiawen, Xie Fengying, Fan Haidi, et al. Classification for dermoscopy images using convolutional neural networks based on region average pooling [J]. IEEE Access, 2018, 6: 65130-65138. [20] Tajbakhsh N, Shin JY, Gurudu SR, et al. Convolutional neural networks for medical image analysis: Full training or fine Tuning? [J]. IEEE Trans. on Medical Imaging, 2016, 35(5): 1299-1312. [21] Szegedy C, Liu Wei, Jia Yangqing, et al. Going Deeper with Convolutions [C] //Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston: IEEE, 2015: 1-9. [22] Deng Jia, Dong Wei, Socher R, et al. ImageNet: A large-scale hierarchical image database [C] //Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Miami:IEEE, 2009:5206848. [23] 陈灿, 包佳怡, 胡粟. 影像组学和深度学习在胰腺癌诊疗和评估中的研究进展[J]. 中国生物医学工程学报, 2023, 42(3): 353-359. [24] 李航, 余镇,倪东,等. 基于深度残差网络的皮肤镜图像黑色素瘤的识别[J]. 中国生物医学工程学报, 2018, 37(3): 274-282. [25] 黄晓鸣, 何富运, 唐晓虎, 等. U-Net及其变体在医学图像分割中的应用研究综述[J]. 中国生物医学工程学报, 2022, 41(5): 567-576.
