Prostate cancer (PCa) is one of the most common and deadly cancers worldwide. A critical step in managing it is Gleason grading, which assesses how aggressive the tumor is by examining the glandular structures in tissue samples under a microscope.
The Gleason grade groups patients into five prognostic categories based on the two most predominant architectural patterns seen in the tissue. Accurate grading is essential for guiding treatment, but the process is time-consuming, labor-intensive, and subject to variability between pathologists due to PCa's biological complexity.
The field of pathomics converts digitized whole-slide images (WSIs) of tissue into analyzable data using artificial intelligence, linking visible tissue patterns to clinical outcomes. While deep learning (DL) models have shown strong performance in this area, they often function as uninterpretable 'black boxes,' which limits their acceptance by clinicians.
This study set out to identify which specific image-derived features are most critical for distinguishing cancerous from non-cancerous tissue and for separating low-grade from high-grade tumors, using an approach that prioritizes transparency and interpretability over raw model performance.
The study used a publicly available dataset of 187 annotated H&E-stained whole-slide images from prostatectomy specimens. Each image was annotated by experienced pathologists to label tissue regions as Gleason pattern 3 (G3), pattern 4 (G4), pattern 5 (G5), normal prostatic tissue, or stroma.
To standardize the images before analysis, Macenko normalization was applied. This technique uses color deconvolution to correct for differences in staining intensity across slides, ensuring consistent color representation and more reliable feature extraction.
Images were analyzed at three magnification levels: 5x, 10x, and 20x. Each image was divided into non-overlapping tiles of 512 x 512 pixels, and only tiles with at least 90% tissue coverage were retained. This multi-scale approach captures tissue architecture at different levels of detail, from broad glandular patterns down to cellular structure.
From each tile, 1,971 features were extracted across four categories: histogram-based features (capturing intensity distributions), Gray Level Co-occurrence Matrix (GLCM) texture features, wavelet features (capturing multi-scale texture), and Local Binary Pattern (LBP) features (describing local microstructure). Features were derived from multiple color channels including RGB, HSV, and separated Hematoxylin and Eosin stains.
To reduce redundancy among the extracted features, a Spearman correlation filter was applied, removing highly correlated features. Statistical significance was then tested using the Wilcoxon rank-sum test, and the final feature set was selected using the LASSO algorithm, which retains only the most predictive features.
Two binary classification tasks were defined: distinguishing Malignant vs. Non-malignant tissue and classifying High-grade vs. Low-grade tumors. Five machine learning models were trained: Gradient Boosting Machine (GBM), Linear Discriminant Analysis (LDA), Multi-layer Perceptron (MLP), Random Forest (RF), and eXtreme Gradient Boosting (XGB).
To prevent overfitting and ensure fair evaluation, data were split so that tiles from the same patient appeared in either the training set or the test set, but never both. A 5-fold patient-level cross-validation was used during training, and a separate hold-out test set provided final performance metrics.
SHAP (SHapley Additive exPlanations) was used to interpret model predictions. Based on game theory, SHAP assigns each feature a contribution score for every prediction, revealing which features drove each classification decision and in which direction -- making the otherwise opaque machine learning models transparent and clinician-accessible.
For the Malignant vs. Non-malignant classification task, all five models achieved very high performance, with AUC values ranging from 0.97 to 0.99 across all three magnification levels. This confirms that handcrafted pathomic features can reliably distinguish cancerous from non-cancerous prostate tissue.
The High-grade vs. Low-grade classification task proved considerably harder, with maximum AUC values of only 0.71 to 0.73 (achieved by MLP and LDA models). This lower performance reflects the inherent biological similarity and heterogeneity between Gleason patterns 3 and 4+5, which present more subtle texture differences.
Confusion matrix analysis confirmed these trends: the diagnostic task showed very few misclassifications across all models, while the grading task had a higher proportion of errors, particularly at lower magnifications. Visual inspection of misclassified tiles revealed that errors often occurred in regions with ambiguous glandular architecture or reduced tissue density.
A quantitative analysis of misclassified tiles found that GLCM, LBP, and wavelet features differed significantly between correctly and incorrectly classified tiles. This suggests these texture-related features are particularly important for correct prediction, and their absence or ambiguity contributes to classification errors.
SHAP analysis revealed that for the Malignant vs. Non-malignant task, the most consistently important features across all three magnification levels were R_or_fo_Median (median intensity in the red channel), R_wavLL_fo_SD (wavelet-based standard deviation), and S_or_fo_90P (90th percentile of saturation). Their appearance at all magnifications suggests they capture robust, scale-invariant tissue properties related to cellularity and tissue composition.
For the High-grade vs. Low-grade grading task, wavelet-derived features were consistently among the most important, appearing prominently across multiple models and magnifications. LBP (Local Binary Pattern) features also contributed significantly, confirming their value in capturing subtle tissue microstructure differences that differentiate tumor grades.
Some features were important only at specific magnifications, indicating that each scale provides unique diagnostic information. For example, E_wavLL_glcm_DV (derived from the Eosin stain channel) was a key feature exclusively at 5x magnification, while other features were relevant only at 10x or 20x. This justifies the multi-scale approach.
Simple histogram features -- such as the 90th percentile, median, and variance of intensity values -- were key for the diagnostic task. Their effectiveness despite their simplicity may reflect differences in cellularity and overall tissue density between malignant and non-malignant regions, aligning with human visual interpretation of stained tissue.
The study's pathomic approach is positioned as a transparent, lightweight alternative to deep learning. While DL models like those in the PANDA Challenge achieve near-pathologist agreement on large datasets, they require massive annotated datasets, significant computational resources, and are difficult to interpret. The present framework performs well with only 187 cases and offers clear, actionable insights into which features drive predictions.
By combining multiscale handcrafted features with SHAP-based interpretability, this study provides direct associations between image characteristics and tissue morphology. This transparency is important for clinical translation, as pathologists are more likely to trust and adopt AI tools when they can understand the reasoning behind a prediction.
The study's results align with prior work showing that wavelet and color features are relevant for PCa grading. However, this work expands beyond previous studies by integrating a broader feature space (multiple color channels, four feature types, three magnifications) and applying SHAP systematically across all models and tasks -- a combination not previously reported for this dataset.
The study acknowledges key limitations: the relatively small dataset of 187 patients may limit generalizability, and external validation on independent cohorts has not yet been performed. Future work will focus on acquiring larger multi-institutional datasets. Additionally, the rarity of Gleason pattern 5 tissue (from only 5 patients) prevented its independent classification.
This study successfully identified the most discriminative pathomic features for automated prostate cancer diagnosis and Gleason grading from H&E-stained whole-slide images, using a multi-scale, multi-channel approach combined with SHAP-based interpretability.
Wavelet features and Local Binary Pattern descriptors proved consistently important for distinguishing tumor grades, while simpler histogram features were effective for the diagnostic task of separating malignant from non-malignant tissue. Features consistent across magnification levels indicate robustness as tissue descriptors, while magnification-specific features highlight the complementary value of analyzing tissue at multiple scales.
The framework demonstrates that transparent, interpretable machine learning built on handcrafted features can achieve high diagnostic performance and provide new insights for pathologists -- even with limited data -- making it a practical tool for resource-constrained clinical settings.