Detection of malignancy in whole slide images of endometrial cancer biopsies using artificial intelligence.

PLoS One 2023 AI 6 Explanations View Original
Original Paper (PDF)

Unable to display PDF. Download it here or view on PMC.

Plain-English Explanations
Pages 1-2
Automating Pathology Workload Triage

An endometrial biopsy is the key diagnostic procedure for suspected endometrial cancer. Tissue obtained from the biopsy is processed into H&E-stained glass slides that are examined by pathologists, who categorize each case as malignant, benign, or insufficiently sampled. As pathology departments increasingly digitize their slides into whole slide images (WSIs), the volume of digital images requiring review is growing rapidly.

This study addressed a practical workflow problem: the vast majority of endometrial biopsies are benign, yet malignant slides requiring urgent clinical decisions are intermingled throughout pathologists' queues. An AI model that automatically sorts slides and flags malignant cases could prioritize which slides get reviewed first - speeding time to diagnosis for patients with cancer without delaying benign cases meaningfully.

The proposed three-category classification - malignant, other or benign, or insufficient - mirrors the categories actually used in clinical pathology reporting. The 'insufficient' category represents slides where the biopsy sample contained too little tissue to make any diagnosis - an important clinical distinction from slides that are adequately sampled and benign.

Unlike most published studies that classify cancer subtypes or predict molecular features, this work focused on the more fundamental task of cancer detection from real NHS clinical samples. The dataset came directly from four NHS Glasgow hospitals, reflecting the genuine variation in tissue preparation and staining protocols found in everyday clinical practice rather than curated research datasets.

TL;DR: AI-based malignancy detection in endometrial biopsy slides could automatically prioritize malignant cases in pathologists' queues, reducing time to diagnosis for cancer patients.
Pages 2-5
Dataset and Annotation Strategy

The dataset comprised 2,909 whole slide images from endometrial biopsies collected by four NHS Greater Glasgow and Clyde hospitals. The malignant category included adenocarcinoma (518 cases), carcinosarcoma (83 cases), sarcoma (25 cases), and hyperplasia with atypia (226 cases) - the latter included because it is a high-risk precancerous lesion. The benign category covered five tissue states (hormonal, inactive/atrophic, proliferative, secretory, and menstrual endometrium), with 182 cases classified as insufficient.

A distinctive feature of this dataset is its intentional variability: slides came from four hospitals each with independent tissue fixation, processing, and staining protocols, and tissue sections were cut at two different thicknesses using four different H&E staining protocols - creating eight different lab combinations. Two complete labs were held out for the test set to explicitly measure generalization to tissue prepared differently from the training data.

Annotation was performed by biomedical scientists and pathologists using QuPath software, following a novel strategy designed for the complex structure of endometrial biopsies. Rather than annotating only malignant regions (the approach used for breast cancer datasets), each slide received an overall class label with only regions that differed from the overall class requiring specific annotation. This was necessary because some slides consisted of many tiny tissue fragments - annotating each fragment individually would have been impractically time-consuming.

The annotation challenge was compounded by the presence of blood and mucus on many slides - non-diagnostic material that needed to be excluded from analysis. A random forest classifier was trained specifically to detect blood and mucus regions on a pixel-by-pixel basis, using color and texture features. Tissue detection separated tissue from slide background. These preprocessing steps together identified which slide regions contained diagnostically relevant tissue.

TL;DR: 2,909 clinically collected endometrial biopsy slides from four NHS hospitals were annotated for malignancy with a novel slide-level labeling strategy, incorporating deliberate lab variation for generalization testing.
Pages 6-9
Three-Stage AI Processing Pipeline

The complete pipeline has three stages that process each slide sequentially. First, tissue detection identifies which regions contain actual tissue (excluding background, blood, and mucus). Second, patch classification divides tissue into patches and trains a convolutional neural network to predict whether each patch is malignant or benign. Third, slide-level classification aggregates patch predictions into a single slide diagnosis of malignant, other or benign, or insufficient.

For patch classification, the GoogLeNet (Inception v1) CNN architecture was trained on approximately 600,000 tissue patches extracted from the training slides. Three patch sizes were experimentally compared: 256 x 256, 512 x 512, and 1024 x 1024 pixels at 40x magnification. The largest patch size was ultimately selected as it gave the best balance of overall accuracy and malignant sensitivity, providing pathologists with enough tissue context to distinguish between malignant and benign cellular patterns.

Patch predictions were assembled into probability heatmaps - visual maps showing the predicted probability of malignancy at every location on the slide. These heatmaps serve two purposes: they provide interpretable visualizations showing exactly where the model detected cancer, and they serve as input to the slide-level classifier. The heatmap approach mirrors how pathologists view slides - first scanning for suspicious areas, then evaluating those areas in detail.

Three slide-level classification algorithms were compared: a feature extraction approach based on a published breast cancer algorithm, a custom XGBoost classifier tuned with features specific to endometrial tissue, and a CNN trained on the heatmaps directly. The XGBoost approach tuned for endometrial biopsies achieved the best balance of overall accuracy and malignant class sensitivity, and was selected for the final model.

TL;DR: A three-stage pipeline of tissue detection, patch-level CNN classification, and heatmap-based slide-level XGBoost classification progressively converts whole slide images into malignancy predictions.
Pages 12-16
Final Model Performance

The final model used 1024 x 1024 pixel patches with the XGBoost slide classifier tuned to prioritize detection of malignant cases. On the test set - containing slides from two labs never seen during training - the model achieved 89.7% overall accuracy and 96.6% correct classification of malignant slides. This near-identical performance to the validation set (88.9% accuracy, 96.9% malignant sensitivity) demonstrates the model generalizes to tissue from different hospital laboratories.

The tuning for high malignant sensitivity came at a modest cost: by weighting the malignant class more heavily during training, overall accuracy dropped from 91.1% to 88.9% but malignant slide sensitivity rose from 90.1% to 96.9%. The authors explicitly chose this tradeoff because missing a cancer is clinically more serious than incorrectly flagging a benign slide, which would simply require pathologist review rather than delayed diagnosis.

Interobserver agreement among three pathologists who independently reviewed 295 test slides showed kappa values of 0.95 at the category level and 0.89 at the subcategory level - excellent agreement that confirms the slide labels used for training were high quality. The main area of pathologist disagreement was between 'insufficient' and 'inactive/atrophic', reflecting a genuine diagnostic challenge that the AI also struggles with.

Analysis of incorrectly classified cases revealed three systematic failure modes: blood and mucus detector errors (tissue misidentified as blood causing tissue to be excluded from analysis), tissue detector failures where slide background was mistaken for tissue, and patching grid misalignment on slides with many small tissue fragments. These identified failure modes provide a clear roadmap for improving the pipeline.

TL;DR: The final model achieved 89.7% overall accuracy and 96.6% malignant slide sensitivity on held-out test slides from different hospital labs, with most errors traceable to specific preprocessing pipeline failures.
Pages 1, 17, 25
Toward Pathologist Workload Prioritization

The clinical application envisioned is not AI replacement of pathologists but AI-driven workload triaging: malignant slides would be automatically moved to the front of the review queue. Since most endometrial biopsies are benign, pathologists currently encounter cancer cases in an unprioritized mix. Sorting slides before pathologist review could compress the time between biopsy and cancer diagnosis from days to hours.

The probability scores output by the slide classifier can be used to rank slides by confidence, not just sort them into binary categories. Even cases the model incorrectly classifies may still rank highly if the classifier is uncertain - as demonstrated by one misclassified case where malignant and benign probabilities were nearly equal (0.455 vs 0.456). Such borderline cases would still receive earlier pathologist attention than slides confidently predicted as benign.

The patch-level heatmaps provide a second clinical benefit: they guide pathologists' visual attention within the slide to the specific regions most likely to contain cancer. Rather than scanning the entire slide, pathologists can focus initial attention on highlighted regions - particularly valuable for slides with many tissue fragments where manual scanning is most burdensome.

The study's use of data from real clinical NHS samples rather than curated research datasets is a notable strength. The deliberate inclusion of multiple staining protocols, tissue thicknesses, and hospital processing pipelines mirrors real deployment conditions, making the performance estimates more credible for clinical translation than studies performed on uniform, high-quality research images.

TL;DR: The AI system is designed to triage malignant slides to the front of pathologist review queues and highlight suspicious tissue regions, with validation on real multi-hospital NHS clinical samples supporting clinical credibility.
Pages 24-25
Limitations and Future Improvements

The main limitation is the difficulty classifying 'insufficient' slides: only 182 such slides exist in the dataset, making this the most underrepresented class. The CNN-based slide classifier correctly classified only 5% of insufficient slides, though the better-performing XGBoost approach performed significantly better. Collecting more insufficient slide examples would substantially improve this aspect of the model.

The hyperplasia with atypia subcategory was disproportionately misclassified compared to adenocarcinoma, suggesting the model has particular difficulty with this category. Hyperplasia with atypia exists on a spectrum - ranging from changes that may spontaneously regress to those progressing to cancer - and even pathologists disagree about borderline cases. This inherent ambiguity in the ground truth makes the task harder for both humans and AI.

The blood and mucus detection step - which currently misclassifies some very pale-stained tissue as non-diagnostic material - could be improved by incorporating stain normalization before preprocessing, or by training the patch classifier to handle blood and mucus patches directly as a third class rather than excluding them from the pipeline entirely.

A parallel weakly supervised approach using multiple instance learning was explored but did not match the fully supervised pipeline's performance. The advantage of weakly supervised methods is that they require only slide-level labels rather than the time-intensive region-level annotations needed for the current model. With a larger dataset, weakly supervised methods may eventually match or exceed supervised performance while requiring far less annotator time - a potentially important scalability advantage for clinical deployment.

TL;DR: Model performance could be improved by addressing blood and mucus detection errors, collecting more 'insufficient' slide examples, and potentially moving toward weakly supervised methods that require less annotator time at scale.
Citation: Open Access, 2023. Available at: PMC9994759.