Prostate cancer is the most commonly diagnosed cancer in men in developed countries, accounting for roughly 19% of new cancer diagnoses in the United States. Detecting it accurately and early is critical, as localized prostate cancer is usually curable while advanced disease is not.
Multi-parametric MRI (mp-MRI) has emerged as a powerful imaging tool for prostate cancer. Unlike standard ultrasound (the older standard), mp-MRI combines several types of imaging data: T2-weighted images showing tissue anatomy, apparent diffusion coefficient (ADC) maps showing how water moves through tissue, and high b-value diffusion-weighted images (BVAL) highlighting areas of restricted water movement -- a sign of dense cellular tissue often associated with cancer.
Together, these three image types provide structural, metabolic, and diffusion information that significantly improves the identification of suspicious regions. However, interpreting these multi-layered images is time-consuming and requires specialized expertise. Even experienced radiologists can miss or mischaracterize lesions.
Automating the detection and outlining (segmentation) of suspicious lesions on mp-MRI could help radiologists work faster and more consistently. This study tests whether a type of deep learning architecture called a Residual Network (ResNet) can learn to identify and delineate prostate lesions as well as expert radiologists.
Mp-MRI scans from 346 patients were obtained from the publicly available Cancer Imaging Archive. For each patient, three radiologists independently reviewed the T2-weighted, ADC, and high b-value images and outlined any lesions that scored at PI-RADS category 3 or higher -- meaning at least equivocal risk for clinically significant cancer.
PI-RADS (Prostate Imaging Reporting and Data System) is the standardized scoring system used worldwide by radiologists to rate the likelihood that a region on prostate MRI represents cancer, on a scale from 1 (benign) to 5 (highly suspicious). Using category 3 and above as the threshold captures the range of clinically relevant lesions.
The segmentations from the most experienced radiologist (a genitourinary specialist with 15 years of experience) were used as ground truth to train and evaluate the model. Segmentations from the other two radiologists were kept separate to provide a human baseline for comparison of inter-observer variability.
From the images, over 93,000 image patches (75x75 pixel squares) were generated, each containing all three imaging channels (T2W, ADC, BVAL). Patches were centered on lesion locations or sampled from non-lesion background areas. The data was split 70% for training (65,000 patches) and 30% for validation (28,000 patches).
A Residual Network (ResNet) was used because these architectures can be trained much deeper than conventional neural networks without the performance degradation that typically occurs as networks get very large. ResNets use shortcut connections that allow information to bypass layers, making gradient flow during training more stable.
The network was trained to perform semantic segmentation -- not just to classify whether a patch contains a lesion, but to assign a cancer probability to each individual pixel. This was achieved using atrous (dilated) convolutions, which expand the network's visual field without reducing image resolution or losing fine spatial detail.
To handle the imbalance between the much larger number of background patches compared to lesion patches, a class-weighted loss function was used, assigning twice the penalty for misclassifying a lesion pixel as background. This encouraged the model to remain sensitive to lesions rather than defaulting to predicting everything as normal tissue.
During training, data augmentation (random rotations and flips) was applied at each epoch to reduce overfitting. Training was stopped automatically after 12 epochs, when validation loss stopped improving, preventing the model from memorizing the training data rather than learning generalizable features.
The ResNet achieved a Hit or Miss accuracy of 93% -- meaning it correctly identified whether a lesion was present or absent in 93% of image patches evaluated on the held-out validation set. This metric directly measures whether the model flags a region for further clinical attention or dismisses it.
When the patch-level predictions were assembled into whole-slice predictions (combining all patches across a full MRI cross-section), the detection accuracy remained at 93% and the Jaccard score -- a measure of spatial overlap between the model's outlined region and the radiologist's -- improved to 71%. A Jaccard score of 71% means roughly 71% of pixels were correctly categorized in terms of being inside or outside the lesion boundary.
The model's area under the ROC curve (AUC) was 97%, indicating excellent overall discriminative ability across different probability thresholds. This performance is competitive with or better than other leading published approaches, which reported AUC values of 94% and 96%.
Notably, the model's lesion outlines agreed more closely with the most junior radiologist (average Jaccard = 73%) than with the expert. This suggests the model tends to outline lesions somewhat conservatively, similar to less experienced clinicians. When compared against biopsy results, the expert radiologist detected 96.1% of clinically significant cancers, while the network detected 82.6% -- a gap that reflects the challenge of automated detection of the smallest and subtlest lesions.
The primary clinical application envisioned is as a decision support tool for radiologists -- not replacing their judgment but helping them work faster and more consistently by flagging regions of concern before they begin their review. This is similar to spell-check for writing: the radiologist remains in charge but benefits from automated prompts.
The PI-RADS scoring system assigns each suspicious lesion a risk score that guides whether a biopsy is recommended. Integrating the model's segmentation output with PI-RADS scoring would add further clinical value and is identified by the authors as a next step. Augmenting the model with patient demographics such as age, PSA levels, and prior biopsy history could also improve performance.
The gap between the expert radiologist (96.1% cancer detection) and the model (82.6%) highlights that automated systems still miss some lesions, particularly small or subtle ones. This reinforces that current AI models in radiology should be deployed as assistants rather than autonomous decision-makers.
A definitive prostate cancer diagnosis still requires a biopsy and histopathological analysis. The model's role is in identifying which regions are most suspicious and warrant biopsy, potentially reducing the number of unnecessary biopsies by better targeting suspicious areas on MRI.
This study demonstrates that a Residual Network trained on multi-parametric MRI data can detect and segment suspicious prostate lesions at a level competitive with state-of-the-art published methods, achieving 93% hit-or-miss accuracy and 97% AUC.
The model learns from three complementary MRI image types simultaneously, mimicking the multi-channel assessment that expert radiologists perform when interpreting prostate mp-MRI. The use of atrous convolutions enables precise pixel-level lesion outlining without sacrificing spatial resolution.
Future work will extend the model to segment distinct anatomical zones within the prostate (such as the peripheral zone and transition zone), which is important because different zones have different cancer risk profiles and require different treatment approaches. Incorporating PI-RADS scores and patient metadata are also planned enhancements.
As mp-MRI adoption grows globally, automated lesion detection systems like this have the potential to reduce the burden on specialist radiologists, standardize lesion assessment, and improve the consistency of prostate cancer detection across clinical settings with varying levels of radiological expertise.