Artificial Intelligence for Renal Cancer: From Imaging to Histology and Beyond

Asian Journal of Urology 2022 AI 8 Explanations View Original
Original Paper (PDF)

Unable to display PDF. Download it here or view on PMC.

Plain-English Explanations
Pages 1-2
Renal Cell Carcinoma and the Case for AI

Renal cell carcinoma (RCC) accounts for approximately 3.0% of all cancer cases worldwide, with an incidence that has been rising by about 2.0% per year. As of 2018, RCC was responsible for roughly 175,000 deaths globally, with around 114,000 in men and 61,000 in women. The highest occurrence is found in North America and Western Europe. The three most common histological subtypes are clear cell RCC (ccRCC), papillary RCC (pRCC), and chromophobe RCC (chRCC), while important benign lesions include angiomyolipomas (0.4% of solid renal tumors) and oncocytomas (3.0% to 7.0%).

A major clinical challenge is that approximately 60% of RCC cases are discovered incidentally through imaging ordered for other reasons. Small renal masses (SRMs), defined as contrast-enhancing lesions up to 4 cm, are increasingly common and typically show slow growth with low malignant potential. However, 10% to 17% of surgically removed kidney tumors turn out to be benign on histopathological evaluation, meaning a significant number of patients undergo unnecessary surgery. Renal mass biopsy (RMB) offers high sensitivity (99.1%) and specificity (99.7%) for detecting malignancy, but it is invasive and carries risks such as hematomas (4.3%) and rare but clinically significant bleeding (0% to 1.4%). Critically, only 64.6% of oncocytomas diagnosed by RMB were confirmed benign after surgical resection, highlighting diagnostic gaps.

The authors argue that AI and machine learning offer a solution by helping to optimize precision and guidance for both diagnostic and therapeutic decisions. This narrative review covers AI applications across the entire spectrum of RCC management: diagnostics through imaging and radiomics, perioperative care including surgical workflow recognition, pathology with histopathological analysis, and long-term follow-up including survival prediction. The most commonly applied models include neural networks, random forest (RF), support vector machines (SVM), and regression models.

TL;DR: RCC makes up 3% of all cancers with rising incidence, and 10-17% of surgically removed kidney tumors are benign. Biopsy has 99.1% sensitivity but misclassifies oncocytomas 35% of the time. This review covers AI applications across RCC imaging, surgery, pathology, and follow-up using neural networks, SVM, random forest, and regression models.
Pages 2-3
Machine Learning Basics for Renal Cancer Applications

The authors provide a primer on how supervised ML works in the RCC context. Data from different aspects of clinical care (imaging, lab results, patient demographics) serve as input. After manual annotation and labeling, ML algorithms are trained to build predictive models. The data are split into training and test sets to prevent overfitting, a common problem where models perform well on training data but poorly on unseen data. Once validated, the final model is applied to classify new cases (e.g., "Is this tumor benign or malignant?") or to predict continuous outcomes (e.g., "How long will the hospital stay be?").

Key algorithms covered include support vector machines (SVM), which find optimal boundaries between data classes; random forest (RF), which builds ensembles of decision trees for robust classification; artificial neural networks (ANN), which learn nonlinear patterns through layered nodes; and deep learning (DL), a subset of neural networks that can automatically extract features from raw data like images without manual feature engineering. DL methods, while powerful, are often criticized as "black boxes" because their decision-making is difficult to interpret, representing a significant hurdle for clinical implementation and medical product regulation.

A critical bottleneck the authors highlight is data preparation. Before AI models can be trained, raw data must undergo pre-processing including segmentation and annotation by human experts. This labeling process is time-consuming and requires attaching metadata such as identifiers, timestamps, and segmentation boundaries. The authors emphasize that an AI model is only as good as the data it is trained on, and the burden of manual annotation remains a major drawback for implementing AI as meaningful clinical assistance.

TL;DR: Supervised ML for RCC requires labeled training data split into training and test sets to avoid overfitting. Key algorithms include SVM, random forest, neural networks, and deep learning. The biggest bottleneck is the time-consuming manual annotation of data by experts, and deep learning's "black box" nature hinders clinical adoption.
Pages 3-5
AI-Enhanced CT and MRI for RCC Diagnosis

Conventional imaging landscape: Multiphase contrast-enhanced CT is the gold standard for RCC diagnosis, with sensitivity of about 90% for detecting renal masses and even higher for lesions greater than 2 cm. MRI offers sensitivities of 86% to 90% and specificities of 76.2% to 93.8% for RCC subtype discrimination. Multi-parametric MRI (mpMRI) has shown 81% diagnostic accuracy for detecting ccRCC and 91% for detecting pRCC, while achieving 100% detection of fat-poor angiomyolipomas with 89% specificity.

Radiomics as a bridge to AI: Radiomics extracts quantitative parameters such as voxel, texture, and histogram analysis from conventional CT or MRI, capturing information beyond what the human eye can perceive. A systematic review and meta-analysis by Muhlbauer et al. reported strong results for discriminating angiomyolipoma from RCC (log odds ratio 2.89) and oncocytoma from RCC (log odds ratio 3.08). In a pooled analysis of 30 studies, the log odds ratio for distinguishing benign from malignant lesions was 3.17 (p < 0.001). Ma et al. demonstrated that radiomics-based CT evaluation outperformed conventional CT analysis in differentiating fat-poor angiomyolipoma from ccRCC across 84 patients.

ML and DL applied to imaging: Li et al. analyzed 61 patients with chromophobe RCC and renal oncocytoma using multiphase CT scans and five ML algorithms: K-nearest neighbors (KNN), SVM, random forest, logistic regression, and multilayer perceptron. All models proved highly accurate, especially when combining data from corticomedullary and nephrographic CT phases. Nassiri et al. presented a radiomic-based ML algorithm tested on 684 patients that achieved an AUC of 0.84 for discriminating benign from malignant renal masses.

Deep learning outperforming radiologists: In a study of 217 patients with pathologically confirmed renal tumors, Xu et al. compared radiomics-based models including random forest and deep learning against radiologists' evaluations. The combination of DL and radiomics achieved an AUC of 0.925, while the pure radiomics model reached an AUC of 0.826. Both outperformed the two radiologists, whose AUCs were 0.724 and 0.667 respectively. A systematic review by Kocak et al. analyzing 30 studies on AI for renal mass characterization highlighted the importance of methodologic quality for clinical integration.

TL;DR: DL combined with radiomics achieved AUC 0.925 for distinguishing benign from malignant renal tumors in 217 patients, outperforming radiologists (AUC 0.724 and 0.667). A radiomic ML algorithm on 684 patients reached AUC 0.84. Five ML models on multiphase CT showed high accuracy for chromophobe RCC vs. oncocytoma discrimination.
Pages 5-6
AI for Surgical Workflow, Augmented Reality, and Tissue Discrimination

Surgical workflow recognition: Nakawala et al. applied the "Deep-Onto" platform to surgical videos of robotic-assisted partial nephrectomy (RAPN). Using more than 700,000 frames derived from nine full RAPN videos, the system defined ten distinct surgical phases. Data were divided into training, validation, and test sets. The trained models achieved 74.0% precision (positive predictive value) and 74.3% accuracy for predicting RAPN surgical steps. Common input data across surgical workflow studies were surgical videos and manual annotation of instruments, fed into artificial neural networks and hidden Markov models.

Predicting operative outcomes: Zhao et al. built ML models (including random forest, regression, and neural networks) to predict operating time for procedures including radical nephrectomy and RAPN. All ML models outperformed the baseline model (which used scheduled case duration and surgeon adjustments), with the authors estimating that accurately planned cases could increase from 35% to over 50%. Bhandari et al. conducted a multi-institutional study on 1,690 RAPN patients with 59 variables, using logistic regression, random forest, and neural networks. The best models achieved AUC 0.858 and precision-recall curve of 0.590 for intraoperative events, and AUC 0.875 and precision-recall curve of 0.706 for postoperative events.

Augmented reality and tissue classification: Nosrati et al. trained a random forest model to recognize color and textural patterns during RAPN, enabling visualization of anatomical structures such as vessels, the kidney, and the tumor. Testing on 15 RAPNs retrospectively yielded a 45% improvement in detection accuracy over prior work. For intraoperative tissue discrimination, Haifler et al. tested Raman spectroscopy on ex vivo specimens of normal kidney tissue and renal carcinoma. Spectra were fed into Bayesian and logistic regression models, achieving 95.8% sensitivity and 88.8% specificity for distinguishing malignant from normal tissue.

TL;DR: Deep-Onto predicted RAPN surgical phases at 74.3% accuracy from 700,000+ video frames. ML models on 1,690 patients achieved AUC 0.875 for predicting postoperative events. Random forest improved augmented reality detection by 45%, and Raman spectroscopy with ML reached 95.8% sensitivity for tissue classification.
Pages 6-7
AI for Detecting and Interpreting Histopathological Features

RCC subtype classification from tissue slides: Holdbrook et al. developed a pipeline to differentiate high-risk from low-risk ccRCC based on histopathologic tissue from 59 patients who underwent surgery. The final classification used a support vector machine (SVM) and achieved F-scores ranging from 0.73 to 0.83 (where 1.0 indicates perfect performance). Tabibu et al. trained convolutional neural networks (CNNs) on hematoxylin and eosin (H&E) whole-slide images from The Cancer Genome Atlas (TCGA), achieving an AUC of 0.98 for detecting ccRCC and AUC of 0.95 for detecting chRCC. They also developed a risk index based on tumor shape and nuclei features that was associated with improved patient survival.

Fuhrman grade prediction: Tian et al. used TCGA data to predict a 2-tiered Fuhrman grade for ccRCC. Seven ML algorithms were trained with nuclei histomics features, including regression with different regularization techniques, neural networks, SVM, and random forest. The final models achieved AUC values from 0.781 to 0.839. Fenstermaker et al. achieved 100% sensitivity and 97.1% specificity with a CNN trained on H&E stained images from TCGA to differentiate normal tissue from RCC. Yeh et al. fed H&E digitized slides into an SVM to detect nuclei for grading ccRCC, reaching an AUC of 0.97.

Deep vs. shallow CNN models: Khoshdeli et al. demonstrated that a deep model CNN outperforms a shallow model CNN when differentiating low-grade granular tumors from high-grade ccRCC on H&E stained images from the TCGA. He et al. took a different approach by using numeric data from marker proteins derived from immunohistochemical (IHC) images of RCC. The K-nearest neighbor (KNN) algorithm linked certain proteins to subtypes of RCC, such as autophagy protein 5 to chRCC.

TL;DR: CNNs on TCGA whole-slide images achieved AUC 0.98 for ccRCC and 0.95 for chRCC detection. SVM-based nuclear grading reached AUC 0.97, and a CNN achieved 100% sensitivity with 97.1% specificity for RCC vs. normal tissue. Seven ML algorithms predicted Fuhrman grade with AUC 0.781 to 0.839.
Pages 7-8
Gene Expression, Methylation, and Molecular Profiling with ML

Gene expression profiling for pRCC staging: Singh et al. used gene expression profiles downloaded from the Genomics Data Commons portal to identify biomarkers that differentiate early from late stages of papillary RCC. Multiple ML algorithms were tested, including random forests, naive Bayes, SVM, KNN, and shrunken centroid classifier. The shrunken centroid classifier and random forest showed the best performances with precision-recall AUC values of 0.812 and 0.815, respectively. In a follow-up study, the same group investigated methylation patterns of pRCC and gene expression from TCGA using similar ML algorithms.

DNA methylation for oncocytoma vs. chRCC: Brennan et al. developed a method to distinguish oncocytoma from chromophobe RCC based on DNA methylation patterns through ML algorithms. This is clinically important because these two entities are notoriously difficult to differentiate by conventional histopathology, yet they require very different management strategies. The methylation-based ML approach could potentially be applied to preoperative biopsy specimens, reducing unnecessary surgeries for benign oncocytomas.

Protein-based classification: He et al. moved beyond morphological features entirely by using numeric data from marker proteins derived from IHC images. The KNN algorithm successfully linked specific autophagy-related proteins to RCC subtypes, such as autophagy protein 5 to chRCC. This protein-based approach offers a complementary pathway to image-based deep learning, combining molecular and morphological information for more precise RCC subtype classification.

TL;DR: Random forest and shrunken centroid classifiers achieved precision-recall AUC of 0.812-0.815 for staging papillary RCC using gene expression data. DNA methylation-based ML distinguished oncocytoma from chRCC for potential preoperative biopsy use. KNN linked autophagy proteins to specific RCC subtypes.
Pages 8-9
Predicting Recurrence, Survival, and Long-Term Outcomes

Recurrence prediction after surgery: Kim et al. performed a comprehensive comparison of eight different ML models on data from 2,814 patients to predict recurrence after surgical treatment of RCC. The models included SVM, logistic regression, decision trees, KNN, naive Bayes, random forest, AdaBoost, and gradient boost. Naive Bayes outperformed all others with an AUC of 0.836 at 5 years after surgery and 0.784 after 10 years. Guo et al. compared a neural network and boosted decision tree model on 697 patients, with the optimized model using predictors including age, sex, tumor laterality, nephrectomy type, T&N status, margin status, and Fuhrman grade to achieve an AUC of 0.877.

Radiomics for mortality prediction: Nazari et al. expanded radiomics beyond diagnosis to predict death in RCC patients. Using CT scans from 70 patients, they trained four classification algorithms: SVM, KNN, generalized linear model, and XGBoost. XGBoost achieved the best performance with an AUC of 0.95 to 0.98, accuracy of 0.93 to 0.98, sensitivity of 0.93 to 0.96, and specificity of approximately 1.0 (reported in 95% confidence intervals). These results demonstrate that radiomics-based AI can predict 5-year mortality risk with remarkable precision.

Metastatic RCC survival prediction: Buchner et al. used clinical and histopathological data from 175 metastatic RCC patients to predict survival at 36 months. Data were prospectively gathered and fed into logistic regression models and artificial neural networks (ANN). In the validation set, the ANN correctly predicted death in 91% of patients with an overall accuracy of 95%, while logistic regression achieved only 78% overall accuracy. This 17-percentage-point advantage for the neural network highlights how nonlinear ML models can capture complex relationships between clinical variables that traditional statistical approaches miss.

TL;DR: Naive Bayes achieved AUC 0.836 for 5-year recurrence prediction across 2,814 RCC patients. XGBoost on CT radiomics reached AUC 0.95-0.98 for predicting 5-year mortality. An artificial neural network predicted metastatic RCC death with 95% accuracy vs. 78% for logistic regression in 175 patients.
Pages 9-10
Barriers to Clinical Implementation and the Path Forward

Data quality and standardization: The authors identify data quality as the most fundamental barrier. AI models are only as good as their training data, yet many studies relied on single-institution datasets with limited external validation. The process of manual annotation and segmentation by human experts remains time-consuming and represents a major bottleneck. Ross et al. proposed self-supervised learning with conditional generative adversarial networks on unlabeled data (porcine nephrectomies from the EndoVis 2017 Challenge) to reduce the required labeled data by approximately 75%, offering a promising solution to the annotation burden.

Black box problem and regulatory hurdles: Deep learning methods are criticized for being difficult to comprehend, which represents a relevant obstacle for clinical implementation, particularly regarding medical product regulations. Health care providers need to develop a basic understanding of AI in order to standardize datasets, define meaningful endpoints, and unify interpretation. The authors emphasize that interdisciplinary collaboration between clinicians, data scientists, and regulatory bodies is essential for bridging this gap.

Overfitting and validation concerns: Overfitting remains a persistent concern across studies, where models show excellent training performance but degrade on unseen test data. Many of the reviewed studies used retrospective designs with limited sample sizes and lacked external validation cohorts. The authors note that for real-world clinical deployment, models need to be validated across multiple institutions, imaging protocols, and patient populations to ensure generalizability.

Conclusions and future directions: Despite these challenges, AI and ML models are evolving rapidly across all aspects of RCC management, from imaging diagnostics through perioperative care to survival prediction. The authors conclude that AI already performs comparably to human counterparts in many tasks. Future implementation requires large and accessible databases with high-quality data incorporating all aspects of RCC care from diagnosis to treatment, enabling external validation and continuous training of AI models. Establishing AI curricula in medical education and fostering interdisciplinary collaboration are identified as essential next steps.

TL;DR: Key barriers include manual annotation bottlenecks, deep learning's "black box" nature hindering regulatory approval, overfitting with small single-institution datasets, and lack of external validation. Self-supervised learning could cut labeled data needs by 75%. The path forward requires standardized multi-institutional datasets, interdisciplinary collaboration, and AI education in medical training.
Citation: Kowalewski KF, Egen L, Fischetti CE, et al.. Open Access, 2022. Available at: PMC9399557. DOI: 10.1016/j.ajur.2022.05.003. License: cc by-nc-nd.