Breast cancer is the most common cancer affecting women worldwide and the second leading cause of cancer death. With 2.3 million new cases recorded globally in 2020, it now exceeds lung cancer as the most prevalent cancer overall. Early and accurate detection remains the most important factor in improving survival outcomes.
Mammography is the primary screening tool for breast cancer, with radiologists examining images for signs such as masses and microcalcifications. However, interpreting mammograms accurately is time-consuming and requires extensive expertise - radiologists with high workloads are prone to fatigue and errors.
These challenges have driven growing interest in computer-aided diagnosis (CAD) systems that use machine learning to automatically analyze mammogram images. Deep learning, particularly convolutional neural networks (CNNs), has emerged as the most powerful approach for this task, with different architectures offering trade-offs between complexity, accuracy, and computational cost.
This study conducts a systematic comparison between shallow CNNs trained from scratch and deep pre-trained CNN architectures adapted through transfer learning, for the task of classifying full mammogram images as benign or malignant.
Most prior work focuses on classifying extracted regions of interest (ROIs) - cropped portions of the mammogram containing a known lesion. This study deliberately works with full-mammogram images, which is a more challenging but clinically realistic scenario that does not require prior manual or automated lesion localization.
Two public mammography datasets are used: CBIS-DDSM (2,704 full mammogram images) and INbreast (410 images). The pre-trained architectures tested include VGG19, ResNet50, MobileNetV2, InceptionV3, Xception, and Inception-ResNet-V2, all fine-tuned on the mammography data.
The first approach uses a custom-built shallow CNN with only two convolutional layers - deliberately small to explore the lower bound of what a simple network can achieve. This model uses binary cross-entropy loss and RMSprop optimization to classify mammograms as benign (label 0) or malignant (label 1).
A key challenge with small medical imaging datasets is overfitting - where the model memorizes training examples rather than learning generalizable patterns. Three progressively refined versions of the shallow CNN were tested: a basic model, one with a dropout layer (randomly disabling 50% of neurons during training to force redundancy), and one also applying data augmentation (flipping, rotation up to 180 degrees, shearing, and scaling).
All images were preprocessed by converting from DICOM format to PNG, normalizing pixel values to the range 0-1, and resizing to 224x224 pixels. The training/test split was 80/20, with model weights saved when validation loss was minimized during training.
Transfer learning allows deep networks pre-trained on large image datasets (like ImageNet's 1.2 million images) to be adapted for medical imaging tasks with relatively small datasets. Rather than training from scratch, the pre-trained model's learned visual features are reused, and only the final classification layers are retrained on the new task.
Six architectures were tested. VGG19 uses 19 layers with small 3x3 filters throughout. ResNet50 uses residual shortcut connections that skip blocks of layers, allowing very deep networks to be trained without gradient degradation. MobileNetV2 is designed for computational efficiency using depthwise separable convolutions, making it suitable for devices with limited processing power.
InceptionV3 applies multiple filter sizes in parallel at each layer to handle objects of varying scales. Xception (Extreme Inception) takes this further using depthwise separable convolutions throughout, while Inception-ResNet-V2 combines the multi-scale Inception approach with residual shortcut connections from ResNet. Each was fine-tuned with additional dense layers and dropout regularization.
Among the shallow CNN variants on CBIS-DDSM, the model with dropout performed best at 80.4% accuracy, while the augmented version slightly underperformed at 79.0%. On INbreast, the dropout model achieved 87.8% accuracy. Data augmentation did not consistently help the shallow model, likely because the small architecture couldn't fully exploit the additional variety.
Among the pre-trained deep CNN architectures on CBIS-DDSM, Xception achieved the highest accuracy at 89.2%, followed by InceptionV3 at 87.6% and Inception-ResNet-V2 at 85.7%. On the INbreast dataset, Xception reached 95.1% accuracy with 100% recall (all true malignant cases identified), followed by Inception-ResNet-V2 at 93.9% and InceptionV3 at 92.7%.
The Xception model achieved the best F1 scores of 89.5% (CBIS-DDSM) and 95.7% (INbreast), reflecting high performance on both precision and recall simultaneously. These results confirm that fine-tuned deep architectures substantially outperform shallow networks trained from scratch across all conditions tested.
Xception's strong performance stems from its use of depthwise separable convolutions, which allow the network to capture spatial and channel-wise patterns independently. This design allows Xception to inherit the residual connection advantages of ResNet while also handling multi-scale features like Inception - a powerful combination for complex medical images.
The INbreast dataset generally produced higher performance than CBIS-DDSM across all models. The authors attribute this to INbreast's higher-quality mammogram images, which contain more discriminating information. The smaller size of INbreast (410 images) was offset by its image quality, demonstrating that data quality can be more important than dataset size when images are sufficiently informative.
Models trained on one dataset and tested on the other performed poorly in cross-dataset experiments. This suggests that the models are learning dataset-specific characteristics alongside clinically relevant features, which is a known challenge in medical AI. Incorporating patient clinical data - age, family history, geographic factors - could further improve robustness and generalizability.
On CBIS-DDSM, the proposed fine-tuned Xception model achieved 89.2% accuracy, matching or exceeding all prior published methods on the same dataset. Prior results ranged from 74.5% (Ansar et al.) to 89% (Tsochatzidis et al.), with the proposed method equaling the prior best while using the full mammogram rather than pre-extracted ROIs.
On INbreast, the proposed Xception model achieved 95.1% - surpassing the prior best of 93.04% (El Houby and Yassin, 2021) and substantially above earlier results ranging from 83.9% to 90%. This improvement is particularly notable given that INbreast is a smaller and more challenging dataset.
The results suggest that when deep networks are properly fine-tuned with appropriate regularization, they can achieve strong results even on full-mammogram images without explicit tumor localization - an important practical advantage since localization requires either manual annotation or additional detection algorithms.
This study conclusively demonstrates that fine-tuned deep CNN architectures substantially outperform shallow networks trained from scratch for breast cancer classification in full mammogram images - confirming that transfer learning from large-scale natural image datasets provides a strong foundation even for specialized medical imaging tasks.
The Xception architecture achieved the best overall performance across both datasets and all evaluation metrics, suggesting it is a particularly effective backbone for mammography classification. Its ability to extract rich multi-scale features from full images makes it well-suited for detecting lesions of varying sizes and appearances.
Future work should incorporate additional clinical metadata, explore transfer learning within the same medical imaging domain rather than from natural images, and test models on more diverse datasets to improve generalizability. Expanding to ROI-level classification and detection could also complement the full-mammogram approach studied here.