This review provides a comprehensive survey of deep learning (DL) applications in breast cancer histopathological imaging, covering three major clinical domains: diagnosis, prognosis prediction, and molecular typing. Breast cancer is the most common malignancy among women worldwide, with over 2.3 million new diagnoses annually. Despite advances in treatment, approximately 30% of early-stage patients experience recurrence within 10 years, underscoring the critical need for better prognostic tools.
The review synthesizes research using foundational architectures including ResNet50, Transformer, and HoVer-Net applied to large-scale datasets from TCGA and multiple clinical centers. It covers the full spectrum of imaging data types: whole-slide images (WSIs), digital breast tomosynthesis, mammography, and MRI, examining how DL automates tumor identification, segmentation, grading, and biomarker assessment.
Beyond diagnostic performance, the review addresses the practical barriers preventing widespread clinical adoption: the shortage of annotated training data, the black-box opacity of AI models that undermines clinician trust, variability across imaging equipment and institutions, and the ethical and regulatory frameworks still needed to govern AI in clinical decision-making. The authors argue that robust, high-quality databases are the single most critical factor in developing DL models with the generalizability required for real-world deployment.
Convolutional Neural Networks (CNNs) remain the dominant architecture for image analysis in breast cancer. ResNet introduces skip connections that allow gradients to bypass layers, addressing the vanishing gradient problem and enabling very deep networks. DenseNet connects every layer to all subsequent layers, promoting feature reuse. U-Net, with its encoder-decoder structure and skip connections between symmetric layers, is the standard for image segmentation tasks including tumor boundary delineation. VGG's simple, consistent 3x3 convolutional structure remains widely used as a transfer learning backbone.
Vision Transformers (ViTs) represent a fundamentally different approach: images are divided into fixed-size patches, flattened into vectors, and processed through self-attention mechanisms borrowed from natural language processing. Unlike CNNs that detect local patterns through fixed convolutional filters, ViTs capture global spatial relationships simultaneously across all patches. Hybrid CNN-Transformer architectures combine local feature extraction from CNNs with global context modeling from transformers, consistently outperforming either architecture alone on complex histopathology tasks.
Generative Adversarial Networks (GANs) address the fundamental data scarcity problem in medical AI by generating synthetic breast images to augment training sets. A GAN consists of a generator network that creates new images and a discriminator that attempts to distinguish synthetic from real images; through iterative adversarial training, generated images progressively become indistinguishable from clinical data. Recurrent Neural Networks (RNNs) process sequential data such as 3D volumetric imaging and time series, while Deep Reinforcement Learning has been applied to landmark detection and lesion segmentation tasks that require sequential decision-making.
Object detection architectures have demonstrated strong performance in identifying and localizing breast tumors across multiple imaging modalities. Faster R-CNN applied to mammography masses achieved recall of 0.9236 and precision of 0.9408 across multiple training sets, improving further to 0.9572 recall when using RGB images. YOLO-based CAD systems trained on DDSM mammograms detected mass locations with 99.7% accuracy and distinguished benign from malignant lesions at 97% accuracy through five-fold cross-validation. A 3D RetinaNet applied to ultrafast DCE-MRI sequences achieved 0.95 sensitivity and 0.90 detection rate across 572 lesions, leveraging both spatial and temporal information to improve small lesion detection.
For lymph node metastasis diagnosis, a comparison study found that DL algorithms substantially outperformed pathologists: the best DL algorithm achieved an AUC of 0.99 while pathologists achieved 0.88, with additional benefits of reduced review time. In histological grading, the Scarff-Bloom-Richardson system requires subjective assessment of glandular formation, nuclear features, and mitotic activity. DL approaches have automated mitosis detection and improved consistency, with the DeepGrade model demonstrating that NHG-2 patients (an intermediate risk group with heterogeneous outcomes) could be reliably stratified into high-risk (HR=2.94) and low-risk subgroups prognostically similar to NHG-3 and NHG-1 respectively.
Thermography-based detection using Inception architectures has achieved remarkable accuracy results in controlled settings. The Inception MV4 model with SGDM optimization reached 100% accuracy at 4 epochs with an AUC of 1 on thermal image datasets, and real-time thermographic video streaming combined with cooling gel application significantly enhanced detection contrast. For a 1 cm tumor at 2 cm depth, cooling gel increased the Inception V3 model's prediction accuracy from 70-89% to 80-94%. These results, while impressive, were obtained in standardized research environments rather than in diverse clinical settings, and thermography remains an adjunct rather than a standalone diagnostic tool.
Tumor-infiltrating lymphocytes (TILs) are immune cells that have migrated into tumor tissue, and their spatial organization carries important prognostic information. DL models have automated TIL quantification from H&E-stained WSIs, finding that high stromal TIL levels were associated with shorter survival (hazard ratio 1.6-2.5 in discovery and validation cohorts), while high TIL levels in triple-negative and HER2-positive breast cancers correlated with better neoadjuvant chemotherapy response (odds ratio 1.28). The smuLymphNet model quantifying germinal centers in axillary lymph nodes found that nodes with an average of 2 or more germinal centers were associated with significantly longer distant metastasis-free survival (HR=0.28) in triple-negative breast cancer patients.
For recurrence risk prediction, a DL model combining WSI analysis of DCIS patients with clinical data achieved 87% accuracy in predicting 10-year ipsilateral recurrence risk by analyzing structural features including annotated normal ducts, cancerous ducts, vascular areas, stromal cells, and lymphocytes. Homologous recombination deficiency (HRD) prediction has emerged as a critical target because HRD, often caused by BRCA1/BRCA2 mutations, determines sensitivity to PARP inhibitors and platinum chemotherapy. DL models analyzing H&E-stained slides achieved AUC of 0.86 for HRD prediction and identified morphological correlates including necrosis, high TIL density, and nuclear heterogeneity, providing a pathology-only alternative to expensive genomic testing.
Multi-omics integration represents the frontier of breast cancer prognosis. In a study of 1,404 invasive breast cancers evaluating 11 biomarkers using DL and multiplex fluorescence immunohistochemistry, five markers (PR, ER, AR, GATA3, and PD-L1) emerged as independent survival predictors (all P less than 0.01). An AI-based radiogenomics approach combining DCE-MRI images with clinical data improved axillary lymph node metastasis prediction to 0.89 accuracy. The spatial transcriptomics algorithm ST-net successfully predicted multi-gene expression levels from WSIs with results closely correlating with RNA sequencing data from TCGA, pointing toward a future where histopathology images serve as a low-cost proxy for comprehensive molecular profiling.
Accurate molecular subtyping into hormone receptor-positive, HER2-positive, and triple-negative breast cancers is essential for treatment selection, yet conventional immunohistochemical assessment is subjective and resource-intensive. DL has achieved strong performance in automating this analysis. The Morphology-Based Molecular Profiling (MBMP) technique predicted hormone receptor expression (ER and PgR) with accuracy rates of 91-98%, comparable to IHC in the majority of patients. A deep neural network multi-instance learning algorithm for HR status assessment achieved AUC of 0.92 with sensitivity 0.932 and specificity 0.741.
The Ki-67 proliferation index, a key marker of tumor growth rate, showed reduced prediction error when DL ensemble models were applied: mean absolute error improved from 0.058 to 0.034 when threshold bias adjustment was incorporated. More importantly, DL-predicted Ki-67 levels were significantly associated with recurrence-free survival (P=0.003) and secondary endocrine resistance (OR=2.49), demonstrating that automated Ki-67 scoring carries genuine clinical prognostic value. For HER2 status, a CNN-based classifier analyzing WSIs achieved AUC of 0.81 for HER2 status prediction and 0.80 for trastuzumab treatment response prediction after 5-fold cross-validation, including accurate identification of HER2-low (1+ or 2+ ISH-negative) tumors relevant to emerging antibody-drug conjugate therapies.
Spatial transcriptomics combined with DL offers a particularly promising frontier: by mapping gene expression data to specific cells and tissue regions in histopathology images, these approaches reveal not just what genes are expressed but where expression occurs within tumor architecture. Studies have successfully predicted TP53 and PIK3CA genomic alterations from WSIs by associating spatial content of slides with genomic mutations, bridging the gap between affordable tissue imaging and expensive molecular testing in settings where genomic profiling is not routinely accessible.
Despite impressive research performance, several persistent challenges prevent DL from achieving widespread clinical deployment. The most fundamental is the shortage of annotated training data. Expert pathologist annotation is time-consuming and expensive, and crowdsourced annotation introduces inter-observer variability. Small sample sizes, class imbalance between rare and common diagnoses, and the absence of standardized annotation protocols collectively limit the scale and quality of training data. Data augmentation and active learning can partially mitigate these constraints but cannot fully substitute for comprehensive, expertly annotated datasets.
The black-box problem is a central barrier to clinical trust. Most DL models categorize samples in binary ways without providing human-readable explanations of their reasoning. Clinicians are ethically and legally obligated to explain diagnostic decisions to patients, and they are reluctant to rely on recommendations they cannot evaluate or justify. Although tools like Grad-CAM, SHAP, and LIME can generate post-hoc explanations, these remain supplementary to rather than integrated within clinical decision workflows. Simultaneously, multi-center generalizability is compromised by variation in scanner models, slide preparation protocols, image formats, and preprocessing algorithms, creating systematic inconsistencies that reduce model performance when applied outside training institutions.
Infrastructure and ethical considerations create additional practical barriers. GPU-based DL training requires specialized hardware and high-cost server clusters, and long-term storage of hundreds of gigabytes of WSIs imposes substantial ongoing costs. Privacy regulations restrict cross-institutional data sharing, though federated learning frameworks offer a pathway to collaborative model training without transferring patient data. Ethical questions around decision-making responsibility remain unresolved: when an AI-assisted diagnosis is incorrect, the allocation of legal and professional accountability between the clinician, the algorithm developer, and the institution requires clear regulatory guidance that existing frameworks have not yet provided.
The most promising near-term advancement is multimodal imaging integration: combining MRI, ultrasound, and digital pathology in a single analytical framework that compensates for the limitations of each individual modality. AI systems integrating multiple imaging sources are particularly valuable in resource-limited settings where specialist availability is low and late-stage diagnosis is common. The authors project that such integrated systems will significantly improve early detection and metastasis prediction for patients who currently lack access to comprehensive imaging facilities.
Federated learning is identified as the critical enabler for building large-scale, demographically diverse datasets without compromising patient privacy. By allowing multiple institutions to collaboratively train shared models while keeping raw patient data local, federated approaches address simultaneously the data scarcity problem and privacy regulatory constraints. This is especially important for rare breast cancer subtypes and underrepresented populations where no single institution can generate sufficient training data alone.
The integration of large language models (LLMs) into breast cancer AI represents the most transformative long-term direction. LLMs pre-trained on massive text corpora can potentially combine tumor characteristics, genomic profiles, clinical history, and patient-specific social factors to generate personalized diagnostic, staging, treatment, and prognostic recommendations in natural language format accessible to both clinicians and patients. However, realizing this potential requires resolving the hallucination problem (LLMs confidently generating incorrect information), establishing standardized clinical benchmarks, and addressing the regulatory and liability questions that AI-assisted clinical decision-making inevitably raises.