Breast invasive ductal carcinoma (IDC) is the most common malignancy of the breast, accounting for the majority of the roughly 306,000 new breast cancer cases diagnosed annually in China alone. By contrast, breast diffuse large B-cell lymphoma (DLBCL) is extraordinarily rare, representing less than 0.5% of all breast malignancies. Both conditions typically present as a painless, palpable breast mass, and conventional imaging modalities including mammography, ultrasound, computed tomography (CT), and breast MRI are unable to reliably differentiate lymphoma from carcinoma based on morphological appearance alone.
The clinical stakes of misdiagnosis: The treatment paradigms for these two diseases are fundamentally different. Breast IDC is primarily managed with surgery, followed by adjuvant chemotherapy, hormonal therapy, radiotherapy, and targeted therapy depending on receptor status. Breast DLBCL, on the other hand, is primarily treated with systemic chemotherapy and radiation, and surgical resection is not the standard approach. When breast lymphoma is mistakenly identified as breast cancer, patients may undergo unnecessary surgery and be deprived of appropriate lymphoma-directed therapy. The literature documents cases of large breast lymphoma nodules containing necrotic tissue being surgically removed after misdiagnosis, with needle biopsy itself contributing to diagnostic error when necrotic tissue is inadvertently sampled.
Current diagnostic limitations: The definitive distinction between these two entities currently requires invasive pathological examination through image-guided needle biopsy or surgical excision, neither of which is ideal. Misdiagnosis rates are elevated partly because breast lymphoma is so rare that many clinicians have limited experience with it. Prior research by Ou et al. demonstrated that radiomic features extracted from 18F-FDG PET/CT scans using traditional machine learning could differentiate breast lymphoma from breast carcinoma, but their study enrolled only 44 patients at a single institution. Machine learning-based radiomics also suffers from poor feature stability and requires labor-intensive manual feature engineering.
This study from Fudan University Shanghai Cancer Center (FUSCC) and Shanghai Proton and Heavy Ion Center (SPHIC) sought to address these gaps by developing and validating an end-to-end deep learning radiomics model using 18F-FDG PET/CT images from 236 patients across two institutions, with the goal of noninvasively and automatically discriminating breast DLBCL from breast IDC.
The study retrospectively analyzed 324 breast nodules from 236 patients (235 female, 1 male; mean age 51.31 years, range 24 to 86) who underwent baseline 18F-FDG PET/CT imaging between January 2009 and December 2021 at one of the two participating centers. The final cohort included 160 breast IDC patients (174 nodules) and 76 breast DLBCL patients (150 nodules). The near-equal lesion count (174 IDC vs. 150 DLBCL) was intentional to provide a balanced training signal, achieved in part because most DLBCL patients present with multiple lesions while IDC patients more commonly have single lesions.
Dataset partitioning: The 249 breast nodules (182 patients) from FUSCC were designated as the internal dataset and split into training and testing cohorts at a 4:1 ratio using stratified sampling, yielding approximately 200 nodules for training and 49 for internal testing. The 75 breast nodules (54 patients) from SPHIC were held out entirely as an independent external validation dataset to assess the model's generalization across institutions, scanner types, and time periods. This two-center design is significantly stronger than single-institution cross-validation designs prevalent in prior radiomics literature.
Eligibility criteria: Included patients were adults (age over 18) with pathologically confirmed breast IDC or DLBCL who underwent baseline PET/CT before any treatment. Exclusion criteria removed patients with other pathological subtypes, prior treatment, other concurrent malignancies, or incomplete clinical data. Across 205 breast cancer patients screened, 45 were excluded, yielding 160 IDC cases. Across 124 breast lymphoma patients screened, 48 were excluded, yielding 76 DLBCL cases.
Clinical characteristics: Patient demographics were well-matched across internal and external datasets, with no statistically significant differences in sex, age, height, weight, or BMI (all P values above 0.05). Disease staging differed between IDC (TNM classification) and DLBCL (Ann Arbor classification), with the distribution of stages significantly different between the two disease groups (P less than 0.001). Nodule size (2D diameter, 3D diameter, and volume) and several PET parameters (SUVmean, SUVmax, metabolic tumor volume [MTV], and total lesion glycolysis [TLG]) were statistically significantly different between IDC and DLBCL in the internal dataset (P less than 0.001 for all), providing evidence that quantitative imaging features carry discriminative information even before the deep learning model is applied.
PET/CT imaging was performed on three different scanner configurations: Biograph 16HR and mCT Flow Siemens systems at FUSCC, and a Biograph 16 Siemens system at SPHIC. All patients fasted for at least 6 hours prior to imaging and maintained blood glucose below 8 mmol/L. Following intravenous injection of 3.7 MBq/kg of 18F-FDG, patients rested for approximately 1 hour before whole-body scanning from head to mid-thigh. CT acquisition used low-dose technique (120 kV, 140 or 150 mA depending on scanner), and PET scans were acquired with 2 minutes per table position, reconstructed with ordered subsets expectation maximization (OSEM) iterative reconstruction using CT-based attenuation correction.
CT image preprocessing: Because the three scanners produced CT images with different slice thicknesses and in-plane resolutions, all CT images were first resampled to isotropic 1 mm3 voxels using bilinear interpolation to ensure inter-scanner comparability. Hounsfield unit (HU) values were then normalized and windowed to a range of -200 to 300, corresponding to the soft tissue window, to suppress irrelevant anatomical information outside the range of interest for breast nodule characterization.
PET image preprocessing: PET scanning intensity was converted from raw count units with absolute activity concentration (Bq/mL) to standardized uptake values (SUV) normalized by patient weight, a standard step that accounts for differences in injected dose and patient body mass. PET images were subsequently resampled to 1 mm3 voxels using bilinear interpolation for consistency with the CT preprocessing pipeline and to reduce scanner-dependent variability that could confound model classification.
Tumor segmentation and patch extraction: All breast nodule regions of interest (ROIs) were segmented on PET images with anatomical guidance from CT by two experienced nuclear medicine radiologists blinded to pathological diagnoses, using ITK-SNAP software. Disagreements were resolved by consensus. From the confirmed ROIs, 2D patches of 128x128 pixels (PET) and 64x64 pixels (CT) were extracted from each tumor slice. Data augmentation techniques including rotation, translation, horizontal flip, vertical flip, and random erasing were applied to increase training set diversity and address class imbalance, with the final model input size standardized to 64x64x1.
The core contribution of this paper is the attention-based aggregate convolutional neural network (AACNN), a custom architecture designed to classify breast nodules as either DLBCL or IDC from 2D PET or CT image patches. The network processes separate inputs from CT and PET imaging modalities in independent parallel streams, each trained to extract modality-specific discriminative features. The architecture contains four convolutional blocks with progressively increasing feature dimensions, drawing inspiration from multiple established design principles including residual connections, bottleneck modules, asymmetric convolutions, and attention mechanisms.
Block-by-block design: The first convolutional block applies a 3x3 convolution followed by switchable normalization (SN) and the Mish activation function. Switchable normalization is an adaptive technique that learns to combine batch normalization, instance normalization, and layer normalization in optimal proportions for each layer, rather than committing to a single normalization strategy. The Mish function (f(x) = x * tanh(ln(1 + e^x))) is a smooth, non-monotonic activation function that outperforms ReLU in regularization behavior. The second block introduces two branches where the main branch applies a bottleneck module and pooling, with 1x1 convolutions inserted to learn non-sparse features. The third and fourth blocks replace traditional n x n symmetric convolutions with asymmetric 1 x n and n x 1 filter pairs inspired by Inception V3, reducing parameter count while preserving representational capacity across channel sizes ranging from 8 to 32.
Convolutional block attention module (CBAM): Between the third and fourth blocks, the AACNN inserts a CBAM, which applies sequential channel attention and spatial attention to recalibrate feature maps. Channel attention (Mc) uses both average pooling and max pooling passed through a shared multi-layer perceptron (MLP) followed by sigmoid to compute per-channel importance weights. Spatial attention (Ms) uses a 7x7 convolutional kernel applied to average-pooled and max-pooled channel descriptors to identify which spatial locations are most discriminative. This dual-attention mechanism explicitly suppresses irrelevant features and highlights the tumor regions most informative for classification, making the model more robust and interpretable.
Ensemble strategy via distance weight voting: Separate AACNN models were trained on CT patches (AACNN_CT) and PET patches (AACNN_PET) independently. The final ensemble model (AACNN_E) fuses their prediction probabilities using a distance weight voting rule: when both models predict the same class (both probabilities above 0.5 or both below 0.5), the higher-confidence prediction is selected; otherwise, the prediction whose probability is farther from the decision boundary (0.5) is selected. This strategy exploits the complementary information in CT anatomy and PET metabolism without requiring a learned fusion layer, keeping the fusion logic transparent and computationally lightweight.
Model training was conducted on a Windows 10 workstation with an Intel Core i7-8750H CPU and an Nvidia GeForce GTX 1060 GPU. Convolutional weights were initialized using the He uniform variance scaling method, which is designed specifically for layers with ReLU-family activations and prevents vanishing gradients in deep networks. L2 regularization with a weight decay coefficient of 0.0001 was applied to limit the squared magnitude of kernel weights and reduce overfitting. Dropout was applied to the fully connected layer to randomly drop nodes during training, further improving generalization. Training used the Adam optimizer with a dynamic learning rate of 0.001, a batch size of 64, and a maximum of 1,000 iterations with early stopping triggered after 20 consecutive epochs without improvement on the validation set accuracy.
Improved binary cross-entropy loss with label smoothing: A notable methodological contribution is the modification of the standard binary cross-entropy loss function using a label smoothing strategy. Label smoothing, drawn from the NeurIPS 2019 work by Muller et al., addresses the problem of erroneous or uncertain labels in the training set by preventing the model from becoming overconfident on its training predictions. The modified loss applies the standard cross-entropy formula when the sample matches its ground-truth class, but applies a smoothed penalty of 0.1 epsilon (set to 0.1) multiplied by a simplified cross-entropy term for the other class. This reduces the risk of the model memorizing noisy labels and improves calibration of the output probabilities, which is particularly valuable in a medical dataset where some labels may reflect ambiguous or borderline cases.
Repeated random subsampling validation: Rather than a fixed single train-test split or k-fold cross-validation, the authors used multiple repeated random subsampling validation, which randomly generates multiple independent splits and averages performance across them. This approach allows better control over the training-to-testing ratio and produces more stable performance estimates than a single split, while also enabling the computation of confidence intervals for each metric. The final reported metrics for the internal testing dataset represent aggregated results across these repeated subsampling iterations, providing more robust performance estimates than single-split evaluations commonly reported in the radiomics literature.
Seven evaluation metrics were computed: accuracy (ACC), sensitivity (SEN), specificity (SPE), positive predictive value (PPV), negative predictive value (NPV), F1 score, and area under the ROC curve (AUC). In this context, sensitivity (SEN) measures the proportion of DLBCL lesions correctly identified as DLBCL, and specificity (SPE) measures the proportion of IDC lesions correctly identified as IDC. PPV reflects the likelihood that a DLBCL prediction is truly DLBCL, while NPV reflects the likelihood that an IDC prediction is truly IDC.
The ensemble model AACNN_E achieved the strongest overall performance. On internal testing, AACNN_E produced an AUC of 0.886, accuracy of 83.0%, sensitivity of 80.9%, specificity of 85.0%, PPV of 84.8%, NPV of 81.2%, and F1 of 0.828. These results were superior to both individual component models across five of the seven metrics (AUC, ACC, SPE, PPV, F1), with the remaining two (SEN and NPV) close to the best individual model. This pattern demonstrates that the ensemble fusion successfully leverages the complementary information captured by the CT-based and PET-based streams.
CT versus PET individual model performance: On internal testing, AACNN_CT outperformed AACNN_PET across all seven metrics, achieving AUC 0.886 vs. 0.831, ACC 82.2% vs. 76.5%, SEN 82.4% vs. 78.1%, and SPE 82.0% vs. 74.7%. This finding suggests that CT-derived morphological and density features are more discriminative than PET metabolic features for this specific classification task, at least with the current preprocessing and patch extraction approach. The superiority of CT is notable given that PET parameters such as SUVmean, SUVmax, MTV, and TLG showed significant univariate differences between IDC and DLBCL in the cohort-level analysis.
External validation performance: On the independent external dataset from SPHIC, AACNN_E achieved AUC 0.788, ACC 71.6%, SEN 61.4%, SPE 84.7%, PPV 84.0%, NPV 62.6%, and F1 0.709. As expected, performance dropped compared to internal testing, with the most pronounced reduction in sensitivity (80.9% to 61.4%) and NPV (81.2% to 62.6%). This pattern indicates that the model was better at correctly identifying IDC (high specificity maintained at 84.7%) than at identifying DLBCL in the external dataset, which may reflect the smaller sample of DLBCL cases available for external evaluation. Pairwise DeLong's test confirmed statistically significant differences between all three models' AUC values (all P less than 0.05).
Statistical validation of predictions: Univariate analysis of the prediction probability values produced by all three models confirmed that DLBCL lesions received significantly higher probability scores than IDC lesions in both the internal and external datasets (all P less than 0.001). For AACNN_E, mean prediction probabilities were 0.34 (IDC) vs. 0.72 (DLBCL) internally and 0.36 (IDC) vs. 0.61 (DLBCL) externally, confirming that the model's outputs are well-calibrated classifiers of disease type rather than random noise.
A key concern with deep learning models in clinical applications is their "black box" nature: high accuracy may be achieved by learning spurious correlations rather than biologically meaningful features. To address this, the authors generated class activation maps (CAMs) visualizing the contribution of different spatial locations in the input image to the model's classification decision. The CAMs were generated across multiple layers, from shallow (activation_1) to deep (block4_agg_1), allowing visualization of how the network's feature representations evolve from low-level texture patterns to high-level semantic features.
Interpretation of activation maps: In the CAM visualizations, brighter (redder) pixels indicate regions that contributed more strongly to the classification, while darker (bluer) regions contributed less. Crucially, the regions highlighted by the AACNN's attention mechanisms corresponded to the actual lesion areas that nuclear medicine clinicians focus on during diagnostic review. This alignment between model attention and clinically relevant anatomy suggests that the AACNN has learned diagnostically valid features rather than imaging artifacts or non-lesion regions. Spatially diffuse attention patterns in DLBCL lesions versus more focal patterns in IDC lesions are consistent with the known biological differences between these entities, with DLBCL typically showing a more infiltrative growth pattern.
Contextualizing performance against the literature: Prior to this work, the only published non-invasive method for distinguishing breast lymphoma from breast carcinoma used traditional machine learning on manually extracted PET/CT radiomic features by Ou et al. (2019, 2020), with 44 patients from a single institution. The AACNN_E model improved AUC by approximately 5% over those prior models while using a substantially larger and more diverse two-center dataset. The AACNN also eliminates the need for manual feature engineering, automatically learning the most discriminative image representations through end-to-end training. The 3D spatial data (volume and 3D diameter) added in this study provided statistically significant discriminative information beyond conventional 2D diameter measurements, contributing additional evidence that spatial tumor characteristics differ between IDC and DLBCL.
The clinical utility of this model extends beyond academic interest. If validated prospectively, an automated noninvasive classifier running on routinely acquired PET/CT scans could flag cases as likely breast lymphoma before invasive workup, prompting more targeted biopsy strategy, appropriate specimen handling, or immediate oncologic consultation for lymphoma. This would be particularly valuable in centers with limited access to expert hematopathologists, where misdiagnosis rates are higher.
Sample size constraints: The most significant limitation is the relatively small number of breast DLBCL cases (76 total, 56 internal and 20 external). Breast DLBCL is an inherently rare disease, and assembling even this cohort required over a decade of retrospective data collection from two major cancer centers. The small external DLBCL sample (20 nodules) is particularly limiting and likely explains the pronounced sensitivity drop on external testing (80.9% to 61.4%). Patients with incomplete clinical data were excluded, further reducing available cases and potentially introducing selection bias. Data augmentation was used to partially compensate for the limited training set, but synthetic augmentation does not fully replicate the diversity of real-world imaging variation.
Input modality and fusion approach: The current AACNN processes CT and PET patches independently with post-hoc probability fusion via distance weight voting. The authors note that PET/CT fusion images combining both modalities in a single input might yield better results than treating them as separate streams. This is biologically motivated: the spatial correspondence between metabolic activity (PET) and structural features (CT) within the same anatomical region contains joint information that sequential processing does not fully exploit. Future work could incorporate true multimodal input fusion, such as early fusion (concatenating PET and CT channels before the first convolutional block) or intermediate fusion using cross-modal attention mechanisms.
Traditional radiomics integration: The study exclusively used deep learning features and did not attempt to combine the learned AACNN representations with handcrafted radiomics features (texture, shape, first-order statistics). Hybrid approaches integrating deep features with traditional radiomic signatures have demonstrated performance improvements over either modality alone in several cancer classification tasks. For the specific challenge of distinguishing breast DLBCL from IDC, combining AACNN output probabilities with quantitative PET parameters (SUVmax, MTV, TLG) or textural features from CT could further improve discrimination, particularly in the external validation setting.
Future directions: Prospective multicenter validation across a broader range of institutions, scanner vendors, and geographic patient populations is the most critical next step before clinical translation. Larger DLBCL cohorts, possibly through collaborative multi-institutional databases or federated learning frameworks that preserve patient data privacy, are needed to retrain and validate models with sufficient statistical power. Additionally, integrating clinical variables (such as LDH, stage, and performance status) with imaging features in a multimodal prediction framework represents a natural extension that could further improve both diagnostic accuracy and prognostic stratification. The authors also highlight that combining deep learning with traditional radiomics features extracted from PET/CT may improve robustness and interpretability beyond what the current AACNN achieves alone.