Prostate cancer is the second most common malignant tumor in men worldwide. Magnetic resonance imaging (MRI) has become a cornerstone of prostate cancer evaluation, but interpreting MRI scans accurately is demanding. Prostate tissue occupies a small area on scans, varies in shape and size between patients, and sits adjacent to organs that can obscure boundaries.
Multi-parametric MRI (mp-MRI) combines several complementary imaging sequences in a single exam: T2-weighted images showing anatomical detail, diffusion-weighted images (DWI) reflecting water movement in tissues, apparent diffusion coefficient (ADC) maps derived from DWI, and Ktrans images capturing blood vessel permeability from dynamic contrast-enhanced sequences.
Each MRI modality captures a different aspect of tissue biology. Cancerous prostate tissue typically restricts water diffusion (low ADC), shows rapid contrast uptake (high Ktrans), and appears hypointense on T2. Reading these complementary signals simultaneously is cognitively demanding and time-consuming for radiologists, and performance varies across skill levels and fatigue states.
Deep neural networks (DNNs) have demonstrated diagnostic accuracy exceeding experienced radiologists in several cancer types including lung and breast. This study proposes a CAD system applying DNNs to mp-MRI to both locate prostate cancer within the gland and classify suspicious lesions as cancerous or benign.
The study used the publicly available PROSTATEx database from Radboud University Medical Centre, a benchmark dataset from the SPIE-AAPM-NCI Prostate MR Classification Challenge. The training set contained 330 suspected lesions from 204 patients, with 76 confirmed cancer lesions and 254 confirmed non-cancer lesions. The test set contained 208 suspected lesions from 104 patients.
Four imaging sequences were included per patient: T2-weighted images in coronal, sagittal, and transverse planes; diffusion-weighted images; apparent diffusion coefficient (ADC) maps; and Ktrans maps. Lesions were labeled within four prostate zones: the peripheral zone, transitional zone, anterior fibromuscular stroma, and seminal vesicle.
Data preprocessing addressed three identified problems: inconsistent sequence acquisition, different image resolutions and grayscale distributions between sequences, and insufficient sample size prone to overfitting. Solutions included standardizing pixel spacing by resampling, normalizing histogram distributions for each sequence, extracting prostate tissue as a region of interest, and applying image augmentation including flip, pan, rotate, zoom, and Mixup to expand the training dataset.
B-spline registration was used to align all four MRI sequences for each patient so that the same lesion location in each sequence was precisely co-registered, allowing the multi-modal network to analyze corresponding tissue regions from all imaging perspectives simultaneously.
The system uses two independent deep neural networks. The first is a prostate cancer localization network based on V-net, a 3D convolutional architecture originally designed for volumetric medical image segmentation. The network takes multi-parametric MRI as input and outputs a 3D segmentation map identifying the spatial location of suspected cancer tissue within the prostate.
The second component is a classification network that determines whether a localized region is cancerous or benign. Three versions were tested: a single-modal network processing one MRI sequence at a time, an input tensor multi-modal network that stacks all five images from one patient as a combined input, and an integrated multi-modal network based on ensemble learning that runs separate sub-networks for each imaging modality and fuses their outputs.
The single-modal and multi-modal networks are built on a lightweight architecture combining concepts from Inception-V3 and VGG-16. They process input through a multiscale stage using 3D convolutions of different filter sizes to detect tumors of varying dimensions, a processing stage using max-pooling and convolutional filter banks for feature extraction, and a dense classification stage to produce a binary cancer or non-cancer prediction.
Training used the ADAM optimizer at a learning rate of 1e-5 with 200 epochs and batch size 4. A focal loss function was applied to address the imbalanced dataset where non-cancer lesions outnumbered cancer lesions. Focal loss down-weights easy-to-classify samples and focuses training on difficult borderline cases, which is critical for medical classification where misclassification of hard cases has the greatest clinical consequence.
The prostate cancer localization network achieved an average localization error of only 1.64 mm, approximately 6 percent of the average prostate volume. This level of precision is clinically meaningful given that the prostate averages roughly 40 x 30 x 20 millimeters in size.
On the full test dataset, the localization network achieved sensitivity of 0.92, specificity of 0.90, positive predictive value of 0.91, negative predictive value of 0.93, and Dice Similarity Coefficient (DSC) of 0.84. The DSC of 0.84 reflects strong geometric overlap between the predicted tumor regions and the radiologist-labeled ground truth.
Compared against five classical medical image segmentation methods, the localization network outperformed all of them. It surpassed U-Net by 0.12 in sensitivity and 0.07 in specificity, and achieved a Jaccard index of 0.89 compared to the next best score of 0.87 from DenseNet and FCN.
The qualitative results, visualized as 2D slices of 3D volume predictions, showed that the predicted tumor regions closely matched the radiologist labels across all four tested patients, with errors remaining below 3 mm in every case.
Among single-modal classification networks, Ktrans performed best with an AUC of 0.853 and 85% accuracy, followed by ADC at AUC 0.826 and 80.5% accuracy. The three T2-weighted image orientations (coronal, sagittal, transverse) all performed around AUC 0.74-0.78, consistently below the functional imaging modalities.
The input tensor multi-modal network, which combined all five image types as a single stacked input, improved AUC to 0.900 and accuracy to 86 percent, outperforming the best single-modal (Ktrans) by approximately one percentage point in average performance metrics.
The integrated multi-modal classification network achieved the best overall performance with an AUC of 0.912, TPR of 0.95, TNR of 0.82, F1-score of 0.8920, and accuracy of 88.5 percent. This network surpassed the input tensor approach by 1.2 percent in AUC and demonstrated greater stability with a 95 percent confidence interval width of only 0.004.
An ablation study on modality combinations showed that using all three modality groups (Ktrans, ADC, and T2-weighted together) achieved the best results, confirming that each imaging type contributes complementary information. Models using two modality groups consistently underperformed the full three-modality combination.
The integrated multi-modal classification network achieved an AUC of 0.912, improving on all previously published models trained on the PROSTATEx dataset. This represents an 8.2 percent improvement over the Inception-V3 approach (AUC 0.83), a 7.2 percent improvement over XmasNet (AUC 0.84), and a 14 percent improvement over a 3D convolutional neural network approach (AUC 0.80).
The T2-weighted images, though performing poorly in isolation, contribute meaningfully to multi-modal fusion. Since 70 to 80 percent of prostate cancers occur in the peripheral zone where T2-weighted imaging has the greatest diagnostic value, excluding T2 images from the multi-modal system would miss important information for this majority of cases.
The training set size sensitivity analysis showed that accuracy increased sharply from 68.3 percent to 88.5 percent as training samples grew from 50 to 200, confirming that performance stabilized near the current dataset size. This suggests that future expansion with larger annotated datasets could yield further incremental improvements.
The proposed system has direct clinical value as an automated pre-reading tool that can flag suspicious lesions and their anatomical locations for radiologist review, reducing reading time, decreasing missed cancers, and providing an objective second opinion that is not subject to fatigue or inter-reader variability.
The study demonstrates that a DNN-based two-stage system combining tumor localization and multi-modal classification can achieve clinically meaningful accuracy for prostate cancer detection on standard mp-MRI, with AUC 0.912 for classification and 1.64 mm average localization error.
The key architectural innovation is the ensemble multi-modal learning approach that trains dedicated sub-networks for each MRI modality and fuses their outputs, allowing each modality to be processed in its optimal feature space while still benefiting from cross-modal complementary information at the decision level.
Practical limitations include the modest size of the PROSTATEx training set (330 lesions) and the reliance on a single institutional dataset. Prospective multi-institutional validation with diverse acquisition protocols will be necessary before clinical deployment.
Future directions suggested include applying the system to larger datasets, exploring additional MRI sequences, and integrating the localization and classification modules into a single end-to-end trainable framework that can simultaneously learn optimal feature representations for both tasks.