Osteosarcoma is the most common primary malignant bone tumor, accounting for approximately 44% of primary malignant tumors in orthopedics. The survival rate for patients with advanced disease remains below 20%, making early and accurate diagnosis a pressing clinical priority. MRI is the preferred imaging modality because it detects aberrant signals in early lesion stages, supports multidimensional imaging, and reveals relationships between soft tissues and adjacent neurovascular structures. It can also quantify bone marrow cavity involvement, information that directly shapes surgical planning and prognostic assessment.
The manual workload problem: Each osteosarcoma patient generates 600 to 700 MRI images per scan session, yet fewer than 20 of those images typically contain diagnostically valid tumor slices. Radiologists and orthopedic oncologists must manually sift through this volume, a process that is time-consuming, fatigue-inducing, and prone to inter-reader variability. Long-term, high-intensity reading work degrades discrimination speed and accuracy, and in resource-limited settings, the shortage of experienced specialists compounds this problem further.
Why automated segmentation is challenging: Osteosarcoma MRI images present a specific cluster of technical difficulties. Noise levels vary considerably across acquisition protocols and scanner hardware. Tumor boundaries are frequently indistinct because osteosarcoma grows into surrounding soft tissue with heterogeneous density gradients. The tumor's location, shape, structure, and density differ substantially from patient to patient, and even the same tumor can appear differently depending on the imaging sequence used. These properties combine to make automated delineation of the lesion region substantially harder than in other oncological domains where boundaries are cleaner.
This paper introduces OSTransnet, a segmentation method built on the Transformer and U-Net architectures, targeting the three core failure modes of prior approaches: noise sensitivity leading to overfitting, semantic gaps in multi-scale feature fusion, and blurred edge segmentation. The authors evaluate the model on a dataset of more than 4,000 osteosarcoma MRI images and compare it against six established segmentation architectures.
The literature on AI-assisted osteosarcoma diagnosis falls into three broad categories. The first is histopathological image classification. Ahmed et al. used a compact CNN architecture with a regularization component to distinguish viable tumors, necrotic tumors, and non-tumor tissue on balanced datasets. Fu et al. developed DS-Net, which combines a depth model with a Siamese network to address the chronic problem of small-dataset overfitting in osteosarcoma classification. Anisuzzaman et al. applied pretrained CNN features to build an automatic classifier of tissue images, improving condition prediction without requiring annotation-heavy training from scratch.
Segmentation algorithms: Several groups have focused directly on delineating the tumor region in imaging. Nasir and Obaid proposed KCG, combining K-means clustering, Chan-Vese segmentation, and iterative morphological operations to achieve high-precision segmentation on existing datasets. Huang et al. introduced MSFCN, adding a supervised output layer to capture both local and global image features. Zhang et al.'s MSRN added three supervised side-output modules to separately extract shape and semantic features. Shuai et al. designed W-net++, cascading two U-Net networks with multi-scale inputs and deep adaptive supervision. All of these approaches improved on simpler baselines but struggled with MRI-specific problems such as high noise levels and indistinct tumor margins.
Treatment response prediction: Beyond diagnosis, AI has been applied to predict chemotherapy response in osteosarcoma. Kim et al. compared multiple methods for predicting neoadjuvant chemotherapy response, informing decisions about whether to continue treatment. Dufau et al. used support vector machines to predict treatment effect before chemotherapy begins, using pretreatment radiological features. Hu et al. developed a CSDCNN algorithm for MRI image recognition that outperformed SegNet and LeNet. Ho et al. used a deeply interactive learning (DIAL) approach to train CNNs as labeling systems for predicting necrosis rates, a key prognostic factor in osteosarcoma.
Limitations of prior work: The authors note that existing MRI-based segmentation methods consistently struggle with two issues. First, the high noise characteristic of osteosarcoma MRI degrades model accuracy and causes overfitting when training data is limited. Second, the wide range of tumor shapes and the blurred boundaries between tumor and surrounding tissue mean that standard skip-connection architectures such as vanilla U-Net leave semantic gaps that reduce edge segmentation precision. OSTransnet is designed to address both deficiencies simultaneously.
OSTransnet's methodology begins before any model architecture decisions are made, with a preprocessing step specifically designed for MRI noise. The fundamental problem is that osteosarcoma MRI images are highly susceptible to noise, and yet simply discarding noisy labeled images would waste annotated data that took significant clinical effort to produce. The authors introduce a data alignment method that turns natural noise into a controlled augmentation resource rather than treating it as contamination to be removed.
Natural noise augmentation pipeline: The process starts by subtracting validly labeled images from their corresponding noisy counterparts to isolate the noise component. A noise clustering technique then groups noise pixels based on ground-truth intensity values. Within each cluster, a random permutation swaps the spatial positions of noise elements. The displaced noise is then recombined with the valid, ground-truth labeled image to generate a new synthetic noisy MRI. This approach produces additional training samples that contain both authentic image content and realistic noise distributions, broadening the training set without introducing artificial artifacts that differ structurally from real clinical images.
The rotation augmentation strategy adds another dimension to data expansion: images are rotated by 90, 180, and 270 degrees before being fed into the segmentation network. During inference, the segmentation probabilities for all four orientations (original plus three rotations) are combined as a weighted average to produce the final segmentation map. This test-time augmentation reduces orientation bias and improves robustness for tumors that appear in varied anatomical orientations across the dataset.
The dataset used for experiments was provided by the Center for Artificial Intelligence Research at Monash University, comprising more than 4,000 osteosarcoma MRI images. The training set accounted for 80% of this data, with 20% reserved for testing. A total of 200 training epochs were used, with the final network parameters determined by joint optimization of convolutional and CTrans (Channeled Transformer) components.
The OSTransnet segmentation model integrates four components: a modified U-Net backbone without skip connections, a Channeled Transformer module (CTrans), an Edge Enhancement Module (BAB), and a combined loss function. The core architectural innovation is the replacement of U-Net's standard skip connections with CTrans, which resolves a well-documented limitation of the skip-connection design.
Why modify U-Net's skip connections: In the original U-Net, skip connections directly concatenate features from the encoder path to the corresponding decoder layer. This approach preserves spatial detail but creates a semantic gap, because the encoder and decoder features operate at different levels of abstraction at the same spatial resolution. When these semantically mismatched features are concatenated, the decoder must reconcile inconsistent representations, which degrades segmentation quality at boundaries and across scale variations. CTrans replaces these direct connections with a cross-fusion transformer mechanism that explicitly bridges the semantic gap through multi-scale channel attention.
CTrans: Channel Cross-Fusion Transformer (CCT) component: CTrans consists of two submodules. The first is the Channel-wise Cross Fusion Transformer (CCT), which fuses multi-scale encoder features using a Transformer's long-range dependency modeling. CCT tokenizes features from all four encoder levels, setting patch sizes to P, P/2, P/4, and P/8 respectively, generating tokens Ti where i = 1 through 4. These tokens are fed into a multi-head channel cross-attention mechanism (not standard spatial self-attention) that operates along the channel axis rather than the spatial patch axis. This channel-axis attention is followed by a multilayer perceptron (MLP) with residual connections. The authors set N (number of attention heads) and L (number of transformer layers) both to 4 after comparing performance at 2, 4, 8, and 12 layers, finding that 4 layers and 4 heads achieved state-of-the-art performance on the dataset.
CTrans: Channel-wise Cross-Attention (CCA) component: The second CTrans submodule handles the decoder side. CCA takes the transformer output from each CTrans level (Oi) and the corresponding U-Net decoder feature map (Di) and passes them through a global average pooling layer to incorporate spatial context. A single linear layer with sigmoid activation constructs a channel attention map that recalibrates and excites the transformer output features, avoiding the dimensionality reduction artifacts that can arise from two-layer bottleneck designs. This mechanism allows CTrans to filter semantically inconsistent decoder features through channel-level cross-checking, which the authors describe as rethinking transformer self-control from a channel rather than spatial perspective.
The third architectural component of OSTransnet is the Boundary Attention Block (BAB), which targets the specific failure mode of blurred edge segmentation. In osteosarcoma MRI, tumors infiltrate surrounding tissue with gradual rather than sharp intensity transitions. Standard segmentation models trained with region-based loss functions, such as standard Dice loss, optimize for volumetric overlap and will tolerate boundary imprecision as long as overall pixel classification is accurate. BAB is introduced to force the network to explicitly attend to edge information as a parallel output pathway.
BAB structure and mask edge extraction: The final decoder feature maps D1 through D4 from the U-Net path are fed as inputs to BAB. A mask edge extraction algorithm first traverses each pixel (i, j) of the segmentation mask and marks as edge pixels any locations where the pixel value is zero and at least one neighboring pixel in the 3x3 grid is non-zero. This generates a mask edge map Mi that explicitly encodes the lesion boundary. The edge map is then convolved and concatenated with the complementary feature map from the previous BAB layer using upsampling. This concatenated feature set is passed through an attention module that produces channel-and-space-compressed weighting vectors, which are multiplied with the input feature map pixel-by-pixel to generate the final boundary-attended output.
Combined loss function: OSTransnet uses a combined loss L that integrates region-based Dice loss with edge-based Boundary loss. Dice loss measures the volumetric overlap between predicted and ground-truth segments, defined as twice the intersection divided by the sum of predicted and actual pixel counts. Boundary loss, formulated using a bounded level set representation with a signed distance map from the ground truth, specifically penalizes predictions that are spatially distant from the true boundary. The combined loss L uses balance coefficients alpha and beta that update through self-learning during training. Early in training, Dice loss receives a higher weight (alpha is larger) to ensure the network learns the gross tumor volume correctly. As training progresses and the network shifts into edge refinement, Boundary loss receives a higher weight (beta grows), directing gradient signals toward boundary precision. This adaptive weighting scheme allows region learning and edge learning to proceed on appropriate timescales.
The interaction between BAB and the combined loss function is by design: BAB provides the architectural pathway for edge-specific feature processing, while the boundary loss term provides the training signal that pushes the network to use that pathway effectively. Neither component achieves the same result in isolation, as the ablation experiments in Section 4 confirm.
Performance was evaluated using six metrics: Intersection over Union (IoU), Dice Similarity Coefficient (DSC), Accuracy (ACC), Precision (Pre), Recall (Re), and F1-score. These metrics collectively capture both volumetric agreement and boundary precision. The study compared OSTransnet against FCN-16s, FCN-8s, PSPNet, MSFCN, MSRN, FPN, and U-Net on the osteosarcoma MRI dataset.
Primary results (Table 3): FCN-based models showed weak recall, with FCN-16s reaching only 0.882 and FCN-8s only 0.873 despite competitive accuracy scores. This pattern indicates that FCN models are prone to under-segmenting the tumor, a clinically dangerous failure mode. PSPNet recorded the lowest IoU at 0.772. MSFCN and MSRN improved substantially in all metrics, with recall rates reaching 0.9, but fell short of clinical targets. U-Net, the strongest baseline, achieved an IoU of 0.867 and a DSC of 0.892. OSTransnet exceeded all baselines: IoU 0.904, DSC 0.949, Precision 0.924, Recall 0.981, and F1 0.951. The DSC improvement over U-Net is 0.949 minus 0.892, approximately 6.4 percentage points. The IoU improvement over U-Net is approximately 4.3 percentage points (0.904 vs. 0.867).
Ablation study (Table 2): The authors tested three configurations to isolate the contribution of each component. The baseline OSTransnet without dataset optimization or BAB achieved IoU 0.889 and DSC 0.931. Adding dataset optimization (without BAB) improved IoU to 0.896 and DSC to 0.938. The full OSTransnet with both dataset optimization and BAB achieved IoU 0.904 and DSC 0.949. These incremental gains confirm that both the noise augmentation pipeline and the edge enhancement module each contribute independently to the final performance, and their effects are additive rather than redundant.
Training stability: Across 200 training epochs, the accuracy of OSTransnet stabilized at 98.7%. The accuracy ranking at convergence was OSTransnet, followed by U-Net, FPN, MSRN, and MSFCN in that order. Recall curves showed that MSRN and MSFCN fluctuated substantially in the first 120 epochs before stabilizing, while OSTransnet maintained consistently high recall throughout. F1-score variation over training was lowest for OSTransnet compared to all baselines, indicating greater training robustness and less susceptibility to mode collapse or gradient instability during learning.
The discussion section provides mechanistic explanations for the performance gaps observed in Table 3. FCN models are designed for semantic segmentation in natural images and lack the capacity to adapt to the high noise and heterogeneous scale variations of osteosarcoma MRI. Their tendency to misclassify normal tissue as tumor (false positives) and to under-segment at boundaries (low recall) reflects both the absence of noise mitigation strategies and the use of fixed receptive fields ill-suited to tumors that vary in size across patients.
PSPNet and FPN limitations: PSPNet uses pyramid pooling to capture multi-scale context, and FPN uses a feature pyramid with lateral connections, but both architectures still rely on spatial feature aggregation that does not explicitly model channel-level semantic relationships. The result is that tumor subtleties at different scales, particularly small satellite lesions adjacent to the main tumor mass, are frequently missed. The authors attribute PSPNet's particularly low IoU (0.772) to its sensitivity to background complexity in MRI images, where the pyramid pooling modules integrate too much non-tumor context.
Why U-Net is the strongest baseline: U-Net's encoder-decoder structure with contextual information integration allows it to suppress complex background interference in MRI, explaining its superior overall performance among the baselines. However, the authors explicitly identify U-Net's architectural limitation: the standard skip connections create a semantic hierarchy inconsistency that reduces sensitivity to both multi-scale tumors and fine edge details. OSTransnet's CTrans module specifically targets this limitation, and the quantitative gap (6.4% DSC improvement) reflects the severity of the semantic gap problem in the osteosarcoma domain.
Limitations acknowledged by the authors: Despite its advantages, OSTransnet still has difficulty fully capturing local information at all scales because the channel cross-attention model, while effective for global dependencies, has limited sensitivity to highly localized features. The authors also note that the small sample size of the dataset remains a constraint on model performance, especially for rare osteosarcoma subtypes that may be underrepresented. The current model was trained and tested on a single institutional dataset, which limits generalizability to other centers with different MRI acquisition protocols or scanner hardware.
OSTransnet represents an advance in the auxiliary diagnosis pipeline for osteosarcoma, a cancer where the combination of high imaging volume, complex tumor morphology, and scarce specialist resources creates a tractable problem for AI intervention. The segmentation model's 98.7% training accuracy and DSC of 0.949 on more than 4,000 images establish a performance baseline that is meaningful for clinical use. The visualized segmentation outputs are designed to give clinicians a clear delineation of the lesion boundary, supplementing rather than replacing their judgment by reducing the manual identification burden.
Practical impact in resource-limited settings: The authors explicitly frame this work in the context of developing countries, where osteosarcoma mortality rates exceed those in high-income settings due to limited medical resources, fewer specialist physicians, and delayed diagnoses. A model that can pre-screen 600-700 MRI images per patient and highlight likely lesion slices could dramatically reduce the diagnostic bottleneck and enable earlier initiation of treatment. Earlier treatment is associated with better response to neoadjuvant chemotherapy and improved candidacy for limb-salvage surgery, both of which influence long-term survival.
Future technical directions: The authors identify two specific directions for future development. First, they plan to incorporate multi-scale segmentation methods that more effectively handle the slight gray-scale differences between tumor tissue and surrounding normal tissue, a subtlety that current architectures handle imperfectly. Second, they intend to integrate additional information sources beyond single-sequence MRI, which could include multi-parametric MRI sequences, diffusion-weighted imaging, or dynamic contrast-enhanced protocols. These extensions would move the model toward a more comprehensive imaging AI that reflects the multi-sequence workflow used in actual clinical osteosarcoma assessment.
Broader significance: Beyond osteosarcoma, the OSTransnet architecture demonstrates a generalizable approach to medical image segmentation in high-noise, edge-ambiguous domains. The combination of natural noise augmentation, channel-axis transformer attention replacing skip connections, and adaptive Dice-Boundary loss could be adapted to other bone tumors such as chondrosarcoma or Ewing sarcoma, or to other anatomical sites where tumor boundaries are similarly indistinct. The paper thus contributes both a validated clinical tool and a methodological template for future work in musculoskeletal oncology imaging AI.