The universal number of women diagnosed with breast cancer is estimated to be around three lakhs in 2026, of which 16% of women are less than 50 years at the time of diagnosis. The rate of survival is generally influenced by the time stages of the disease, age of the patient, genetic influence and diagnostic accuracy. Though imaging is one of the parameters, the confirmation of diagnosis is generally done by histopathological examination (Biopsy) and management based on Immunohistochemistry (IHC) only. Multi-modality system is preferred over Single-modality systems, as the later has the disadvantage of modality-specific noise coupled with low sensitivity. In this paper, a multimodal deep learning framework integrating digital mammography with Ultrasound images is developed by applying dual stream convolutional neural networks complemented with cross model attention fusion and Mote Carlo (MC) dropout. The dataset consists of both mammogram and ultrasound images, and they have been processed by two algorithmic models which are Modified Relation and Margin-Based Deep Learning Network (MReMarNet) and the Multi modal attention model. The proposed method aims to improve accuracy while maintaining reliability of two different datasets, Mini-DDSM and BUSI. Upon training, the results of the Multi modal attention model have a clear advantage over the MReMarNet model, thereby showing better results with rates of 99.00%, 98.00% and 96.00%, 94.00% for the BUSI and Mini-DDSM datasets respectively.

Breast cancer, Mammography, Ultrasound, BUSI, Mini-DDSM, data sets, Convolutional Block Attention Module