Muscle-invasive bladder cancer is treated with neoadjuvant chemotherapy followed by radical cystectomy. The MVAC chemotherapy regimen -- methotrexate, vinblastine, doxorubicin, and cisplatin -- has been shown to improve survival when given before surgery, and patients who achieve complete pathologic response (no residual tumor at surgery) have substantially better outcomes.
However, MVAC chemotherapy carries significant toxicity including leucopenia, sepsis, mucositis, nausea, and alopecia. Because no reliable method exists to predict individual response to MVAC in advance, some patients endure these side effects without benefiting from the treatment, while simultaneously missing the window of opportunity for alternative therapies as their physical condition deteriorates.
An accurate, non-invasive method for assessing treatment response during or after chemotherapy would serve two purposes: allowing physicians to stop ineffective treatment early in non-responders, and potentially identifying complete responders who might be candidates for bladder preservation rather than radical cystectomy. The current gold standard for response assessment is pathologic examination at the time of surgery, which cannot be used while treatment is ongoing.
The core innovation of this study is the creation of hybrid pre-post image pairs as input to the deep learning network. CT scans taken before and after chemotherapy were both segmented, and corresponding regions of interest were extracted from each. A pre-treatment region and its corresponding post-treatment region were then combined into a single 32 by 32 pixel image pair that the network could analyze jointly.
This design allows the network to simultaneously examine what the tumor looked like before treatment and what it looks like afterward, capturing changes in texture, size, and appearance that may predict whether complete response was achieved. Multiple such pairs were extracted from each lesion, yielding 6,209 hybrid image pairs from the 94 training lesion pairs.
The reference standard was pathologic staging at the time of cystectomy. Cases were labeled as T0 (complete response, no residual tumor) or non-T0 (residual tumor present). In the dataset of 123 patients with 129 cancers, 33% achieved complete response. The test set contained 54 lesion pairs from 42 lesions.
Two blinded radiologists independently reviewed each pre-post CT pair in the test set and rated the likelihood of T0 stage, providing a human performance benchmark for the network to be compared against.
The deep learning convolutional neural network (DL-CNN) was based on the AlexNet architecture. It consisted of two convolution layers, two locally connected layers, and one fully connected output layer that produced a score from 0 to 1 indicating the likelihood of complete response. The architecture was deliberately kept small given the limited dataset size.
Transfer learning was used to address the challenge of limited medical imaging data. The network was first pretrained on the CIFAR-10 dataset, which contains 60,000 natural images of everyday objects. The idea is that low-level features learned from natural images -- such as edges, corners, and textures -- are generic enough to provide a useful starting point for medical image analysis.
Three structural modifications to the base network were evaluated by varying the filter sizes, strides, and padding in the first two convolution layers. Additionally, different layer-freezing strategies were tested, where freezing a layer during fine-tuning preserves the pretrained weights for that layer and prevents them from being overwritten by the smaller medical imaging training set.
Training used 10,000 epochs with periodic evaluation on the validation set to identify the optimal stopping point, typically around 2,000 epochs. Final models were retrained on the combined training and validation sets up to this optimal epoch. Full training took approximately 8.3 hours on an Nvidia GTX 1080 Ti GPU.
The best-performing network configuration achieved an AUC of 0.86. This was DL-CNN-2, a modified architecture with different max pooling parameters in its convolution layers, using CIFAR-10 transfer learning. The base network with transfer learning achieved an AUC of 0.79, compared to 0.73 for the same network with randomly initialized weights.
The two radiologists achieved AUCs of 0.76 and 0.77 respectively. The best DL-CNN configuration outperformed both radiologists, and even the base transfer-learning network performed comparably to expert human readers. The radiomics-based method from a prior study achieved an AUC of 0.77, comparable to both radiologists and the base DL-CNN.
Layer freezing showed a clear trend: freezing more layers progressively reduced performance. Freezing only the first layer slightly improved the AUC to 0.81 compared to the base network with no frozen layers at 0.79. Freezing the first two layers dropped it to 0.78, and freezing three layers dropped it to 0.71. This suggests that the first layer, which captures universal edge-like features, benefits from being preserved, while deeper layers benefit from retraining on the medical imaging data.
At a fixed specificity of 80%, DL-CNN-2 achieved 75% sensitivity and 78.9% accuracy, the best among all tested models. The base network with randomly initialized weights achieved only 41.7% sensitivity at the same specificity, underscoring the value of transfer learning for this small-dataset problem.
Medical imaging datasets are typically much smaller than natural image collections. Training deep neural networks from scratch on small datasets risks overfitting, where the network memorizes training examples rather than learning generalizable patterns. Transfer learning mitigates this by starting from weights that already encode useful visual representations.
The finding that freezing only the first convolution layer improved performance while freezing deeper layers hurt performance aligns with theoretical understanding of what different layers learn. First-layer filters in networks trained on natural images tend to detect basic visual primitives like edges and blobs that are universally useful. Deeper layers encode progressively more domain-specific features that need to be retrained for the target application.
The difference between the best (DL-CNN-2, AUC 0.86) and worst (DL-CNN-3, AUC 0.69) structural modifications was statistically significant, demonstrating that architectural choices for the convolution and pooling layers meaningfully affect classification performance even in networks of the same overall depth. This highlights the importance of hyperparameter tuning even in small network architectures.
A reliable computerized decision support system for treatment response assessment could transform bladder cancer management. If a patient can be identified early in the chemotherapy course as a complete responder, bladder preservation surgery rather than radical cystectomy could be offered, dramatically reducing treatment morbidity and improving quality of life.
For non-responders, early identification would allow discontinuation of toxic and ineffective chemotherapy and transition to alternative treatment strategies before the patient's physical condition deteriorates further. The window of opportunity for alternative therapies may be narrow, making timely response assessment critically important.
The authors plan to expand the dataset by collecting additional T0 and non-T0 cases, which would enable testing of larger and deeper architectures such as GoogLeNet Inception and ResNet. A larger dataset would also allow more robust statistical comparisons between model variants. Using CT images rather than CIFAR-10 natural images for pretraining is identified as a potential avenue to further improve transfer learning performance.
Standardizing CT acquisition parameters, including voxel size and slice thickness, across scans would reduce variability that currently complicates network training. While real-world clinical variability is unavoidable, future efforts to interpolate scans to a uniform voxel size could help the network learn more consistently across different CT scanner settings.