Immunotherapy Has Transformed Lung Cancer Treatment. Immune checkpoint inhibitors have become standard care for locally advanced non-small cell lung cancer (LA-NSCLC) that cannot be surgically removed, following landmark trials showing they significantly prolong survival when given after chemoradiotherapy. However, responses are highly variable - some patients survive for years while others progress rapidly.
Why Prediction is Hard. The challenge is identifying before treatment starts which patients are likely to benefit. Without reliable predictive tools, all eligible patients receive similar treatment, inevitably exposing poor responders to side effects without benefit while potentially undertreating those who might respond to more intensive approaches.
Radiomics as a Non-Invasive Solution. Radiomics - extracting quantitative features from medical images - offers a non-invasive way to characterize tumor biology. However, most radiomics methods extract summary features from the entire tumor, averaging out important spatial variation. A more granular approach called voxel-level radiomics captures feature maps at the resolution of individual imaging voxels, preserving information about spatial heterogeneity within tumors.
The Small Dataset Problem. Deep learning models require large datasets to train reliably, but immunotherapy for lung cancer has only been standard practice for a few years, meaning available datasets are relatively small. This creates a risk that deep learning models trained on these small datasets will overfit - learning patterns specific to the training data rather than general biological principles.
Learning from Non-Immunotherapy Data First. The key innovation is a two-stage training approach that leverages the much larger pool of historical lung cancer imaging data. In the first stage, a Vision-Mamba deep learning model was trained on 339 patients from the RTOG 0617 clinical trial - a large cohort treated with chemoradiotherapy without immunotherapy. This gives the model a foundation for recognizing prognostically relevant tumor imaging patterns.
Fine-Tuning for Immunotherapy. In the second stage, the pre-trained model was fine-tuned using just 37 patients from an immunotherapy cohort. Critically, only the later layers of the network were updated during this phase, while the earlier layers - which had learned general tumor imaging representations - were frozen. This preserves generalizable knowledge while adapting to immunotherapy-specific patterns.
Vision-Mamba Architecture. The model uses a Vision-Mamba framework, which is based on state-space sequence modeling. Unlike standard convolutional networks, Vision-Mamba can efficiently capture long-range dependencies across image data - important for characterizing spatial patterns that extend across tumor regions. Its linear computational complexity makes it practical for processing high-resolution voxel-level feature maps.
Dual-Input Design. The model receives two inputs simultaneously: the raw CT images and the computed voxel-level radiomics feature maps. This dual-input design allows the network to leverage both the visual appearance of the tumor and quantitative textural features at each location, providing complementary information that neither input alone can supply.
What Voxel-Level Radiomics Measures. Standard radiomics extracts features from the entire tumor volume at once, producing a single value for each feature that summarizes the whole tumor. Voxel-level radiomics instead calculates these same features using a sliding window approach, producing a spatial map where each location within the tumor has its own feature value. This preserves information about where within the tumor different textures and intensities are found.
Ninety Features Computed. For each CT slice containing tumor tissue, 90 radiomic feature maps were calculated: 17 intensity features (describing brightness and its distribution) and 73 texture features (describing patterns of spatial variation in intensity). All features followed Image Biomarker Standardization Initiative guidelines, ensuring reproducibility and comparability with other studies.
Feature Selection for Efficiency. Not all 90 feature maps were equally useful. Each map was individually tested for its prognostic ability, and only those achieving an area under the ROC curve greater than 0.6 were retained for model training. This filtering step reduces redundancy, manages model complexity, and ensures that the model learns from informative spatial patterns rather than noise.
Biological Rationale. High intratumoral spatial heterogeneity - meaning the tumor contains regions with very different imaging characteristics - is increasingly recognized as a hallmark of immune resistance. By capturing this spatial complexity at the voxel level rather than averaging it away, the model can potentially decode imaging signatures of immune microenvironment composition that are invisible in conventional radiomics.
Strong Discriminatory Performance. When evaluated in the independent immunotherapy testing cohort of 150 patients - who were completely separate from the training and fine-tuning data - the model achieved a concordance index (C-index) of 0.73 with a 95% confidence interval of 0.63 to 0.82. A C-index of 0.5 represents no better than chance, while 1.0 is perfect discrimination, so 0.73 represents meaningful prognostic ability.
Stable Predictions Across Time Horizons. The time-dependent area under the ROC curve was 0.73 at one year and 0.70 at two years. These values indicate that the model maintained its ability to correctly distinguish patients who would survive from those who would not across clinically relevant follow-up periods. Some gradual decline at later time points was expected and reflects the increasing influence of post-treatment factors like subsequent therapy lines.
Well-Calibrated Predictions. Beyond discrimination, a good prognostic model must also be calibrated - meaning that when it predicts a 70% survival probability, roughly 70% of patients with that prediction should actually survive. Calibration curves for the immunotherapy test set showed acceptable agreement with the diagonal reference line, indicating that the model's predicted probabilities closely matched observed outcomes.
Performance Was Maintained Across Datasets. The C-index was 0.74 in the training set, 0.69 in the non-immunotherapy test set, and 0.73 in the immunotherapy test set - a remarkably consistent performance across datasets that differed substantially in their patient characteristics and treatment approaches. This consistency is the hallmark of a model that has learned generalizable patterns rather than dataset-specific noise.
Clear Risk Group Separation. Patients divided into high-risk and low-risk groups based on the model's median risk score showed dramatically different survival outcomes across all three datasets. In the immunotherapy test set, the high-risk group had a hazard ratio of 4.44 compared to the low-risk group - meaning high-risk patients were over four times more likely to die at any given time than low-risk patients (log-rank p < 0.001).
Consistent Stratification Across Treatment Settings. The hazard ratios for risk group separation were 4.74 in the training set, 4.02 in the non-immunotherapy test set, and 4.44 in the immunotherapy test set. The consistency of these numbers across such different patient populations is strong evidence that the model is capturing fundamental tumor characteristics rather than treatment-specific artifacts.
Clinical Net Benefit. Decision curve analysis confirmed that the model provided net clinical benefit across a range of threshold probabilities in all datasets - meaning that using the model to guide decisions would be expected to produce better patient outcomes than treating all patients the same way or treating none of them. This analysis is important because it translates statistical performance into clinical decision-making value.
What the Model Pays Attention To. Gradient-weighted class activation mapping (Grad-CAM) analysis visualized which tumor regions drove the model's predictions. High-risk patients showed attention concentrated within the tumor with spatial gradients - heavier weight toward the tumor core. Low-risk patients showed more diffuse, uniform attention across the tumor region. This difference in spatial attention patterns suggests the model has learned to detect biologically meaningful intra-tumoral heterogeneity.
Spatial Heterogeneity as a Biological Signal. The model's ability to capture spatially heterogeneous activation patterns within tumors reflects an important biological reality: tumor regions differ substantially in their cellular composition, immune infiltration, and metabolic activity. These regional differences are increasingly recognized as key drivers of immune resistance. By preserving this spatial information, voxel-level radiomics can encode complex biology that whole-tumor averages miss.
Why Transfer Learning Is Necessary. The study explicitly showed that the fine-tuning step on a small immunotherapy dataset alone would be insufficient. The high dimensionality of voxel-level features means complex models are prone to overfitting when trained on small datasets. Pre-training on larger non-immunotherapy data provides a stable starting point - the model enters the fine-tuning phase already knowing how to recognize relevant tumor characteristics, reducing the risk of learning spurious patterns from limited data.
Addressing Clinical Heterogeneity. The immunotherapy and non-immunotherapy cohorts differed significantly in sex distribution, tumor histology, radiotherapy techniques, and tumor volumes. Despite these differences, the model maintained consistent performance across both settings. This suggests that the imaging representations learned from historical data capture tumor characteristics that remain prognostically relevant regardless of the specific treatment received.
Complementing Existing Risk Assessment. Current prognostic tools for locally advanced NSCLC rely primarily on tumor stage and patient performance status - factors that reflect tumor extent and patient fitness but not tumor biology. Voxel-level deep radiomics adds a complementary dimension by quantifying intratumoral spatial heterogeneity from pretreatment CT scans, potentially improving risk discrimination beyond what clinical staging alone can achieve.
Pre-Treatment Risk Stratification. The model's most direct clinical application is identifying high-risk patients before treatment begins, using only a routine pretreatment CT scan. This information could guide decisions about treatment intensity - for example, whether to intensify surveillance schedules, consider clinical trial enrollment, or evaluate eligibility for combination strategies involving additional immunotherapy agents.
Practical Visualization for Clinical Use. The authors envision that the model's continuous risk score could be displayed alongside CT images using color-coded heatmaps derived from Grad-CAM, making the model's reasoning transparent to clinicians. Risk scores could be translated into categorical tiers (Low/Intermediate/High) based on clinically meaningful survival probability thresholds, facilitating discussion in multidisciplinary tumor boards.
Sparing Low-Risk Patients from Overtreatment. An equally important application is identifying patients at low risk who might safely avoid escalated treatment protocols. In an era where combination immunotherapy approaches are being explored in clinical trials, the ability to identify patients who are likely to do well with standard treatment could spare them from unnecessary toxicity.
Limitations Requiring Further Work. The study is retrospective with all the selection biases that implies, uses only 2D image slices rather than full 3D volumes, and lacks key immunotherapy biomarkers like PD-L1 expression and tumor mutational burden that would further improve predictions. Prospective validation in multiple institutions with standardized CT acquisition protocols is essential before clinical implementation.
A Viable Solution to the Small Dataset Problem. This study provides proof of concept that the data scarcity challenge in immunotherapy research can be addressed through cross-dataset transfer learning. By borrowing imaging knowledge from historical non-immunotherapy datasets and adapting it to the immunotherapy context, reliable deep learning models can be developed even when immunotherapy-specific training data is limited.
Voxel-Level Radiomics Captures Unique Information. The spatial resolution of voxel-level feature maps provides information about intratumoral heterogeneity that is invisible in both conventional radiomics and standard CT reading. This information appears to be biologically relevant to immunotherapy outcomes, suggesting that the tumor microenvironment leaves detectable imaging signatures that can be quantified with appropriate computational methods.
The Path to Clinical Translation. Future work should focus on prospective validation in diverse patient populations, integration of molecular biomarkers like PD-L1 and tumor mutational burden, full 3D volumetric analysis, and standardization of CT acquisition parameters across institutions. The ultimate goal is a clinically deployable tool that can be run on routine imaging data and integrated into treatment planning software.
Broader Implications. The cross-dataset transfer learning framework demonstrated here is not specific to lung cancer or immunotherapy. The approach of pre-training on large historical datasets and fine-tuning on smaller contemporary cohorts could be applied to predict outcomes for other novel treatments across oncology, where the challenge of limited training data is a universal barrier to developing reliable prognostic models.