A Novel Deep Learning-Based Mitosis Recognition Approach and Dataset for Uterine Leiomyosarcoma Histopathology

Cancers 2022 AI 8 Explanations View Original
Original Paper (PDF)

Unable to display PDF. Download it here or view on PMC.

Plain-English Explanations
Pages 1-2
Why Mitosis Counting Matters in Uterine Leiomyosarcoma

Uterine leiomyosarcoma (ULMS) is the most common sarcoma of the uterus and one of the most aggressive gynecologic malignancies. It arises from the smooth muscle cells of the uterine wall and carries a high risk of recurrence and death, with resistance to most systemic therapies compounding its poor prognosis. Women in the perimenopausal years are most frequently affected, though the cancer can occur across a wide age range. Predisposing factors include prior pelvic radiation, long-term tamoxifen use, and certain inherited genetic syndromes, though most patients present without any identifiable risk factor.

Histopathological diagnosis: The definitive diagnosis of ULMS relies on microscopic examination of tissue obtained by biopsy or hysterectomy. Pathologists apply a set of diagnostic criteria formalized in the Bell et al. Stanford criteria, which weigh three features: mitotic count per 10 high-power fields (HPF), coagulative tumor cell necrosis, and nuclear atypia. Among these three, the mitotic count is considered the single most critical and diagnostically challenging parameter. A count of 10 or more mitoses per 10 HPF, in combination with at least one of the other features, is required to establish a definitive diagnosis of malignancy.

Clinical differentiation challenge: ULMS must be distinguished from two other uterine smooth muscle tumors that share overlapping histological features: uterine leiomyoma (the benign fibroid, often with mitotically active variants exceeding 10 mitoses per 10 HPF but lacking atypia) and smooth muscle tumor of uncertain malignant potential (STUMP), which shows variable atypia and a mitotic count typically below 10 per 10 HPF. This morphological overlap makes accurate mitosis quantification particularly consequential, since misclassification between these entities has direct implications for the extent of surgery and adjuvant treatment decisions.

The authors note that manual mitosis counting is inherently subjective and observer-dependent. The mitotically active area in a given tumor can vary region-to-region within the same specimen, and low-experience pathologists may miss focal mitotic figures, especially under time pressure when screening many slides in a short period. These sources of variability create a meaningful opportunity for AI-based automation to serve as a quality-check or second-opinion system.

TL;DR: ULMS is the most common uterine sarcoma, diagnosed by three histological criteria in which mitosis count is the most important. Manual counting is subjective and error-prone, motivating an AI-based automated solution. This paper presents the first publicly available ULMS mitosis dataset and a YOLOv4-based detection baseline.
Pages 2-3
What Existed Before This Study: Mitosis Detection in Other Cancers

Automated mitosis detection has been studied extensively in breast cancer histopathology, where several large annotated datasets and competitive benchmarks exist. The field has progressed through two broad generations of methods. The first generation relied on handcrafted features extracted from hematoxylin-and-eosin (H&E) stained slides, including morphological descriptors, color statistics, and texture measurements, fed into classical classifiers such as support vector machines (SVM) or decision trees. While workable, these approaches showed limited robustness across scanners and staining protocols.

Deep learning approaches: The second generation uses deep convolutional neural networks (CNNs) to learn discriminative features directly from pixel data. Several architectures have been applied to breast cancer mitosis detection with strong results. Chen et al. used a two-stage cascade in which a detector first segments candidate regions, which are then classified. Li et al. proposed a region-based pipeline where Faster R-CNN identifies candidate mitoses, followed by refinement via ResNet-50 to reduce false positives. Cai et al. modified Faster R-CNN with ResNet-101 as the feature extraction backbone. Dodballapur et al. combined ResNet-50 and the Xception network for false positive suppression in breast cancer slides.

The ULMS gap: Despite this body of work, no study had applied automated mitosis detection to ULMS, and no publicly available annotated dataset existed for this tumor type. The authors stress that ULMS mitoses present additional challenges compared to breast cancer: significantly greater morphological variation in the mitotic figures themselves, the need to distinguish mitoses from apoptotic bodies and other mimics, and the diagnostic requirement to identify the most mitotically active focal area within the tumor rather than counting across a random sampling field. These differences make direct transfer of breast cancer models to ULMS unreliable without dedicated training data.

TL;DR: Automated mitosis detection is well-studied in breast cancer using CNNs (Faster R-CNN with ResNet-50/101, cascade networks), but had never been applied to ULMS. No annotated ULMS dataset existed, creating a research gap this paper specifically fills.
Pages 3-4
Building the First ULMS Mitosis Dataset: Acquisition, Annotation, and Validation

The dataset was collected at Atia Hospital in Karachi, Pakistan under IRB approval (AGH/IRB/2021/01), with informed consent obtained from all patients. A senior pathologist ("Pathologist A") examined histopathological digital images of ULMS cases using a Best Scope microscope (model BS2030BD) at high resolution, manually selecting microscopic patches that were rich in mitotic activity. From these digitized slides, 150 image patches were extracted, each at a resolution of 1280 x 720 pixels. The patches are referred to as "images" throughout the manuscript, reflecting the fact that they represent cropped regions of interest from whole-slide images rather than entire slides.

Annotation workflow: Annotations were performed using the Roboflow annotation tool, a cloud-based platform that allows pathologists to draw bounding boxes around structures of interest. Pathologist A worked in collaboration with a deep learning engineer to annotate all 150 images by drawing rectangular bounding boxes around every visible mitosis. The dual-expert annotation approach was intended to ensure that both clinical expertise and technical understanding of detection requirements were incorporated into the label quality. The final annotation count yielded 348 individual mitosis objects across the 150 images, representing an average of approximately 2.3 mitoses per image patch.

Inter-observer validation: To validate annotation quality, all images and their annotations from Pathologist A were independently reviewed by two additional pathologists ("Pathologist B" and "Pathologist C"). Any case of disagreement among the three annotators, whether about the presence of a mitosis or its exact bounding box placement, resulted in the discarding of the disputed object or image entirely. This consensus-based quality control is important because inter-observer agreement on mitosis detection is a known challenge in pathology, with variability documented even among experienced specialists.

Training and testing split: The 150 images were divided into 100 training images (containing 240 mitosis objects) and 50 testing images (containing 108 mitosis objects). The dataset was made publicly available to the research community upon reasonable request to the corresponding authors, with a link provided in the published article. This represents the first open-access annotated dataset for ULMS mitosis detection.

TL;DR: 150 H&E image patches (1280x720 px) were collected from a single Pakistani hospital, annotated by 3 pathologists using Roboflow bounding boxes (348 total mitoses), split 100/50 for training/testing. All disagreements led to discarding that annotation, ensuring high-quality consensus labels.
Pages 4-5
YOLOv4: Architecture, Design Choices, and Implementation Details

The authors benchmarked four object detection architectures on their ULMS dataset: R-CNN, Faster R-CNN, single shot multi-box detector (SSD), and YOLOv4. Of these, they selected YOLOv4 as the baseline method for publication based on superior overall performance. The fundamental advantage of YOLO-type detectors over region-based methods like Faster R-CNN lies in their single-pass design: YOLO performs object localization and class prediction simultaneously in one forward pass through the network, whereas Faster R-CNN first generates candidate regions using a separate proposal network, then classifies each region independently. This architectural difference translates to significantly lower inference latency, which matters for clinical workflow integration.

YOLOv4 internal structure: The YOLOv4 architecture used in this study consists of three main components. The backbone is CSPDarknet53, which uses cross-stage partial (CSP) connections that split feature maps into two branches and merge them to improve gradient flow and reduce redundant computation. The neck is a path aggregation network (PANet), which fuses feature maps from different spatial scales, allowing the detector to handle mitotic figures of varying sizes. The head is the standard YOLO detection head responsible for outputting predicted bounding box coordinates and confidence scores. CBL blocks (convolution, batch normalization, leaky-ReLU) and CBM blocks (convolution, batch normalization, MISH activation) serve as the core computational units throughout the network.

Implementation: The network was implemented in Darknet, an open-source framework written in C with CUDA support for GPU parallel processing. This implementation choice offers lower-level hardware control compared to Python-based frameworks such as TensorFlow or PyTorch, and was selected explicitly for its ability to run on both GPU and CPU hardware. Training was performed on a Windows 10 Pro workstation with an Intel Core i9 10th-generation processor (3.30 GHz), 256 GB RAM, and four NVIDIA RTX 3090 GPUs. For the detection task, the network input was resized to 416 x 416 pixels. Since only one object class (mitosis) was to be detected, the number of output filters in the final convolutional layers was set to 18 (calculated as 3 anchors x (5 + number of classes) = 3 x 6 = 18).

Training hyperparameters: The learning rate was set to 0.00065, batch size to 64 with a mini-batch of 16, momentum to 0.949, and weight decay to 0.0005. The learning rate schedule used step decay at iterations 6,400 and 7,200. The training loop ran for 10,000 steps per checkpoint, with the model snapshot showing the best validation performance selected for final testing. Data augmentation techniques were applied to the 100 training images to increase effective training set size, though the specific augmentation types (rotation, flipping, brightness adjustment, etc.) are not enumerated in detail in the manuscript.

TL;DR: YOLOv4 with CSPDarknet53 backbone, PANet neck, and YOLO detection head was selected over Faster R-CNN and SSD after comparative testing. Implemented in Darknet/CUDA on 4x RTX 3090 GPUs. Input: 416x416 px, learning rate 0.00065, batch size 64, 1-class detection with 18 output filters per detection layer.
Pages 5-6
Detection Performance: Precision, Recall, F1-Score, and Comparative Results

The trained YOLOv4 model was evaluated on the held-out test set of 50 images containing 108 annotated mitosis objects. Detection outputs were compared against the ground-truth annotations using an intersection-over-union (IoU) threshold to determine true positives. Of the 108 ground-truth mitoses, the model correctly identified 97 (true positives, TP = 97), missed 11 (false negatives, FN = 11), and generated 33 spurious detections with no corresponding ground-truth object (false positives, FP = 33). The average confidence score for all detections across the test set was 87%, indicating that the model was generally certain about the regions it identified as mitotically active.

Primary metrics: From these raw counts, precision was computed as TP / (TP + FP) = 97 / 130 = 0.7462. Recall was computed as TP / (TP + FN) = 97 / 108 = 0.8981. The F1-score, the harmonic mean of precision and recall, was 0.8151. In the context of pathology diagnostics, high recall is generally prioritized over precision because missing a mitotic figure (false negative) is more clinically consequential than flagging a non-mitotic structure for review (false positive). The recall of 0.8981 indicates the model successfully detected roughly 9 out of every 10 true mitoses.

Statistical robustness: To verify that these numbers were not the result of a single favorable random evaluation, the authors ran the model 10 separate times on the test set and computed descriptive statistics across runs. The results were: precision mean 0.7462 (SD +/- 0.041), recall mean 0.8981 (SD +/- 0.038), F1-score mean 0.8151 (SD +/- 0.035), and accuracy mean 0.6879 (SD +/- 0.053). The low standard deviations across all metrics confirm that the reported performance values are stable and reproducible rather than outliers driven by randomness in inference.

Comparison with competing architectures: YOLOv4 was also compared against SSD and Faster R-CNN on the same test dataset. SSD achieved precision 0.7037, recall 0.8796, and F1-score 0.7819. Faster R-CNN achieved precision 0.7287, recall 0.8704, and F1-score 0.7932. YOLOv4 outperformed both competitors on all three metrics, with improvements of approximately 2-4 percentage points in F1-score relative to SSD and 2 percentage points relative to Faster R-CNN. These margins are meaningful given the small test set size and confirm that the single-pass detection approach of YOLO offers genuine advantages for this specific application.

TL;DR: YOLOv4 on 50 test images (108 mitoses): precision 0.7462, recall 0.8981, F1 0.8151, avg confidence 87%. Across 10 repeated evaluations, SD remained below 0.055 for all metrics. YOLOv4 outperformed SSD (F1 0.7819) and Faster R-CNN (F1 0.7932) by 2-4 F1 points.
Pages 6-7
What the Detection Results Mean for Pathology Practice

The paper situates its results in the broader challenge of pathological grading of ULMS and related uterine smooth muscle tumors. One key clinical context is the distinction between leiomyosarcoma, leiomyoma, and STUMP. Mitotically active leiomyomas, which are benign, can show more than 10 mitoses per 10 HPF but characteristically lack nuclear atypia or tumor necrosis. STUMPs may exhibit focal moderate-to-severe atypia but typically have a mitotic count below 10 per 10 HPF (mean 3-4 per 10 HPF) and absent tumor necrosis. The morphological resemblance between these entities is the fundamental reason why mitosis counting must be accurate: an overcount could lead to inappropriate radical surgery, while an undercount could result in inadequate treatment of a true malignancy.

AI as a second-opinion system: The authors explicitly propose their framework as a second-opinion system rather than a standalone diagnostic replacement. A pathologist would still perform the primary examination, but the AI model would flag mitotically active regions in the image, display bounding boxes around detected mitoses with associated confidence scores (mean 87%), and provide an automated count. This assists in ensuring the most mitotically active focal area of the tumor is identified and counted, which is a requirement of the diagnostic criteria. The visualization in the paper shows detected mitoses highlighted with purple bounding boxes, making them immediately apparent even to technologists without prior pathology training.

Inter-observer variability: A critical implicit argument in the paper is that AI can reduce human variability. The authors note that less experienced pathologists may miss mitotic figures, especially when screening large numbers of slides under time pressure or when mitotic activity is focal and occurs in a small area of an otherwise large specimen. By providing a map of mitotically active regions, the system can direct the pathologist's attention to areas requiring careful scrutiny. This kind of region-of-interest highlighting is distinct from, and potentially more useful than, a simple total mitosis count per HPF.

TL;DR: The model is proposed as a second-opinion tool for ULMS grading. It highlights mitotically active regions with bounding boxes and confidence scores, helping pathologists identify the most active focal areas, reduce observer variability, and disambiguate ULMS from leiomyoma and STUMP, which require different clinical management.
Page 7
Dataset Size, Generalizability, and Methodological Constraints

The most significant limitation acknowledged by the authors is the small size of the dataset. With only 150 annotated image patches from a single institution (Atia Hospital, Karachi, Pakistan), the model has been trained and validated on a narrow sample of cases. Deep learning object detection models generally benefit from thousands of annotated examples to achieve robust generalization, particularly for a task as morphologically variable as mitosis detection. The 348 total mitosis annotations distributed across 150 images represents a starting point, not a production-grade training set.

Single-center, single-scanner bias: All images were acquired using the same microscope model (Best Scope BS2030BD) at one hospital. Histopathological image appearance can vary substantially depending on the tissue preparation protocol, staining batch, scanner model, and magnification settings used. A model trained exclusively on images from one scanner at one center may perform substantially worse when deployed on images from a different scanner or institution, a well-documented problem in computational pathology known as domain shift. No external validation was performed in this study, so the true generalizability of the YOLOv4 model across sites remains unknown.

Annotation methodology: The bounding box annotation format captures the presence and rough location of mitoses but not their precise boundaries. Segmentation-based annotations (pixel-level masks) would enable more precise shape analysis, which might improve detection of partially overlapping or irregularly shaped mitotic figures. Additionally, the dataset was assembled by selecting mitosis-rich patches rather than sampling patches systematically across the entire slide, which could introduce selection bias in the types and density of mitoses represented.

Absence of whole-slide integration: The current framework operates on 1280 x 720 pixel patches rather than on complete whole-slide images (WSIs). In clinical practice, pathologists examine entire glass slides, and an AI system would need to be integrated into a whole-slide image analysis pipeline to function at scale. The paper does not address how the patch-level detector would be applied across a full slide or how the most mitotically active 10 HPF area would be identified programmatically, which is the clinically required measurement.

TL;DR: Key limitations include a small 150-image single-center dataset from one scanner, no external validation, bounding box (not segmentation) annotations, and no whole-slide inference pipeline. The model cannot yet replace a pathologist's slide-level assessment or be considered validated for any clinical setting.
Pages 7-8
Expanding the Dataset and Moving Toward Clinical Integration

The authors identify dataset expansion as the primary next step. The 150-image dataset establishes a public baseline that can be augmented by other research groups. Multi-center collaboration would address both the sample size limitation and the single-scanner domain shift problem by incorporating images from hospitals with different tissue preparation protocols, scanner types, and patient demographics. Because ULMS is a relatively rare cancer, multi-institutional collaboration is particularly important for assembling a dataset large enough to train and robustly validate a clinical-grade model.

Improving the detection model: Beyond dataset expansion, future work could explore more sophisticated architectures. YOLOv4 was state-of-the-art at the time of publication (2022) but has since been succeeded by YOLOv5, YOLOv7, YOLOv8, and transformer-based detection models. Incorporating attention mechanisms could help the network focus on the fine-grained morphological differences between mitotic figures and mimics such as apoptotic bodies, pyknotic nuclei, and lymphocytes. Semi-supervised or weakly supervised learning could also leverage slide-level labels (ULMS diagnosis) to guide patch-level annotations, reducing the annotation burden on pathologists.

Whole-slide integration: The most clinically meaningful extension would be a full whole-slide image (WSI) pipeline that automatically tiles the slide, applies the patch-level detector across all tiles, aggregates detections, and identifies the hotspot region with the highest mitotic density per 10 HPF. This output would directly mimic the pathologist's diagnostic workflow and could be presented in a digital pathology viewer as an overlay on the original slide image. Standardizing the HPF area (0.2 mm2 per field at 40x magnification, 10 fields in the most active area) within such a pipeline is a technical challenge that requires careful calibration for each microscope/scanner combination.

Broader significance: This paper is the first to demonstrate that deep learning can detect mitoses in ULMS histopathology images with meaningful accuracy, and the first to provide an annotated dataset for this tumor type to the public. Even as a baseline, the YOLOv4 F1-score of 0.8151 compares favorably with published results for mitosis detection in breast cancer from similar-scale single-center studies. The publicly released code and dataset enable other groups to build on this work directly, which the authors cite as a primary contribution alongside the detection results themselves.

TL;DR: Future directions include multi-center dataset expansion, newer YOLO/transformer detection models, and development of a whole-slide inference pipeline that identifies the most mitotically active 10 HPF area automatically. This study sets the first public ULMS mitosis benchmark (YOLOv4 F1 = 0.8151) and releases code and data for the community.