Artificial intelligence has already proven itself in radiology diagnosis, matching or surpassing human experts at detecting tumors in breast, lung, and other cancers. The next challenge is using the same imaging data not just to detect cancer, but to predict how a patient's disease will behave and how it will respond to treatment. This perspective article focuses on this more complex application of AI in oncology imaging.
Traditional biomarkers in oncology are mostly molecular tests. Examples include Oncotype DX and MammaPrint in breast cancer, and Decipher in prostate cancer. These genomic assays are expensive, require invasive tissue biopsies, and can only be done at a single point in time. AI-enabled imaging biomarkers offer a compelling alternative: they are non-invasive, inexpensive, and can be repeated serially throughout treatment.
A prognostic biomarker tells clinicians how aggressive a patient's cancer is and what the likely outcome will be, independent of treatment. A predictive biomarker identifies which patients will benefit from a specific therapy. Both types can now potentially be derived from routine radiology scans using AI, enabling more personalized treatment decisions without additional invasive procedures.
AI imaging biomarkers also offer a key advantage over biopsies: they can characterize the tumor across its entire three-dimensional volume, capturing the full spectrum of intratumoral heterogeneity rather than a small needle sample that may miss the most aggressive tumor regions.
Radiomics is the extraction of large numbers of quantitative features from medical images that are invisible to the human eye. These include intensity-based features (such as average pixel brightness or kurtosis, which reflects how non-uniform the intensity distribution is), texture features (which quantify spatial patterns between neighboring voxels), and shape features (measuring tumor geometry in 3D).
Texture features are particularly important because tumor heterogeneity on imaging often reflects underlying biological heterogeneity. For example, the grey-level co-occurrence matrix captures how often pairs of pixels with specific intensity values appear adjacent to each other, providing a fingerprint of tumor tissue architecture that correlates with grade, mutation status, and prognosis.
Shape radiomics goes beyond standard tumor measurement tools like RECIST (which rely on simple 2D diameter measurements) to characterize the full 3D geometry of tumors. Features like surface-to-volume ratio and fractal dimension can detect irregular growth patterns associated with poor outcomes. Studies have shown that changes in tumor volume during treatment outperform standard RECIST criteria for monitoring response.
A growing area is peritumoural radiomics, which extracts features from the tissue surrounding the tumor rather than just within it. The tumor microenvironment (TME) plays a critical role in cancer behavior, and radiomic analysis of the TME has shown predictive and prognostic value across breast, lung, brain, prostate, and other cancers. Including peritumoural features consistently increases predictive power over intratumoural analysis alone.
Deep learning (DL) uses neural networks with many layers to learn directly from image data, bypassing the need for predefined feature extraction. Each layer transforms the input progressively, building increasingly abstract representations until a final prediction is made, such as the probability of cancer recurrence or response to therapy.
Convolutional neural networks (CNNs) are the dominant DL architecture for medical image analysis. CNNs apply filters across the image to detect local spatial patterns, then combine those patterns hierarchically. Unlike radiomics, CNNs are not limited to predefined features and can discover novel imaging signatures that human researchers would not have thought to measure.
Training a DL model requires very large datasets because models may have millions of adjustable parameters. A strategy called transfer learning partially solves this problem: a model pretrained on a large general image database is repurposed and fine-tuned for medical imaging tasks, enabling strong performance even with smaller cancer-specific training sets.
DL models are often combined with radiomics in hybrid approaches. Some studies extract features from both a pretrained CNN and a hand-crafted radiomic pipeline, then combine them in a statistical model. These hybrid approaches frequently outperform either method alone, suggesting that DL and radiomics capture complementary aspects of tumor biology.
In prostate cancer, both radiomics and deep learning applied to multiparametric MRI scans have been used to predict biochemical recurrence after radical prostatectomy. One study achieved an AUC of 0.95 for predicting 3-year recurrence by combining MRI texture features, PI-RADS score, and clinical parameters in a support vector machine model. This substantially outperforms clinical risk scores alone.
Another prostate cancer study used a deep transfer learning model to distinguish indolent from clinically significant cancer on multiparametric MRI, achieving an AUC of 0.726 versus 0.711 for the standard PI-RADS v2 score, demonstrating that AI can outperform current clinical imaging scoring systems for cancer characterization.
In lung cancer, radiomic nomograms combining tumor texture and shape features with clinical data have predicted disease-free survival with C-indexes exceeding 0.72, outperforming TNM staging alone. A 3D CNN model trained to predict two-year overall survival after radiotherapy or surgery achieved an AUC of 0.71, with genomic association studies linking its learned features to cell cycle and transcriptional processes.
In brain cancer, radiomic models using both intratumoural and peritumoural texture features from multiparametric MRI, combined with molecular markers like IDH status, achieved C-indexes of 0.84 for predicting progression-free survival in glioblastoma. A DL model integrating multiparametric MRI data could stratify patients into subgroups with expected median overall survival of under 10 months, 10 to 15 months, and over 15 months.
Radiomic and DL models have been applied to predict whether patients will achieve pathological complete response (pCR) to neoadjuvant chemotherapy before surgery, across breast, rectal, bladder, and lung cancers. A large multicentre breast cancer study using multiparametric MRI radiomic features achieved an AUC of 0.79 for pCR prediction, validated across three institutions. Predicting pCR before treatment begins could spare non-responders from toxic chemotherapy.
For targeted therapy prediction, radiomics has been used to identify which patients are likely to benefit from HER2-targeted therapy in breast cancer and EGFR inhibitors in lung cancer, based on pretreatment CT or MRI features linked to the tumor's molecular subtype or mutation status. This is the domain of radiogenomics: using imaging features as non-invasive surrogates for tumor genetic status.
A rapidly growing application is predicting response to immune checkpoint inhibitors (ICIs). Multiple studies have developed radiomic models from pretreatment CT scans to predict ICI response, with AUCs of 0.80 to 0.85 across lung, head and neck, and bladder cancers. Some models can even distinguish true responders from patients who experience hyperprogression, a paradoxical acceleration of tumor growth on immunotherapy that requires immediate treatment cessation.
AI models have also been used to predict microsatellite instability (MSI) status from CT texture features, which is relevant because MSI-positive tumors are particularly responsive to ICIs. One study achieved 92.5% sensitivity and 96.9% specificity for MSI status prediction from colon cancer CT scans, without requiring tumor tissue biopsy for molecular testing.
The most critical challenge is data quality and reproducibility. Most radiomic studies show a sharp drop in performance when moving from the training dataset to independent validation, particularly when tested on data from different institutions with different imaging protocols. The mean quality score in a meta-analysis of 77 radiomic studies was only 26% of the maximum possible, indicating widespread methodological weaknesses.
AI models for outcome prediction require large, carefully curated datasets. Retrospective datasets introduce selection biases that can distort model performance. Randomized controlled trials (RCTs) provide the cleanest data, but accessing RCT datasets is slow and requires complex approvals. Emerging techniques like federated learning, where models are trained across multiple institutions without sharing raw patient data, may enable large-scale validation while preserving privacy.
Interpretability is a major barrier, particularly for deep learning. While hand-crafted radiomic features can sometimes be connected to biological meaning, CNN-derived features are black boxes. When an AI-enabled biomarker drives a treatment decision, clinicians and patients need an explanation grounded in pathophysiology. Research into saliency maps and attention mechanisms that highlight which image regions drive model predictions is actively addressing this gap.
Regulatory and reimbursement pathways also present hurdles. AI imaging biomarkers must be approved by agencies like the FDA in the US and demonstrate clinical utility, not just predictive accuracy, to be incorporated into clinical guidelines. Demonstrating that AI-guided treatment decisions improve patient outcomes, rather than just correlating with them, requires prospective interventional trials that take years to complete.
AI-enabled imaging biomarkers represent a transformative opportunity to extract far more clinical value from the radiology scans already being performed for cancer patients. Rather than relying solely on visual interpretation, AI can systematically measure hundreds of imaging features that correlate with tumor biology, treatment response, and patient survival.
The field is moving from proof-of-concept feasibility studies toward multi-institutional validation and prospective clinical trials. Key initiatives like The Cancer Imaging Archive, multi-institutional collaborative networks, and standardization efforts from organizations like the Image Biomarker Standardization Initiative are building the infrastructure needed for reliable, reproducible AI biomarkers.
The future will likely involve multimodal AI models that integrate imaging data with genomics, pathology, and electronic health records to create comprehensive cancer phenotype profiles. These integrated models could support clinical decisions at every stage of the patient journey, from initial risk stratification through treatment selection, response monitoring, and surveillance for recurrence.
For clinicians, AI imaging biomarkers should be understood as decision-support tools, not replacement decision-makers. Their value lies in surfacing information from images that humans cannot perceive, enabling more data-driven, personalized oncology decisions. The goal is not to replace clinical judgment but to augment it with quantitative, reproducible imaging intelligence.