When radiation therapy targets tumors in the abdomen -- including the prostate, liver, pancreas, and kidneys -- a major challenge is that these organs move as the patient breathes. A tumor that is centered in the radiation beam at the start of treatment may shift by more than a centimeter during delivery, risking both underdosing the tumor and overdosing healthy tissue.
4D-CT (four-dimensional CT) addresses this by acquiring multiple CT scans timed to different phases of the breathing cycle. The result is a series of 3D images, typically ten, that capture the organ's position at different respiratory stages -- essentially a movie of the patient's internal anatomy breathing in and out.
To make full use of 4D-CT data, the different phase images must be precisely aligned with each other through a process called deformable image registration (DIR). Accurate DIR enables motion tracking of tumors and fiducial markers (small gold seeds implanted near the tumor), realistic target volume definition that accounts for motion, dose accumulation across treatment sessions, and respiratory gating (delivering radiation only when the tumor is in the correct position).
Abdominal 4D-CT registration is among the hardest registration problems in radiation therapy. The abdomen has large, complex motion patterns, low image contrast between soft tissues, and the large image size places extreme demands on both computation speed and algorithm accuracy.
Conventional intensity-based deformable image registration methods such as optical flow and demons work by iteratively adjusting a deformation field until the two images become as similar as possible. While accurate, these methods are very slow -- taking minutes to hours for a single 4D-CT dataset -- and must be repeated many times during treatment planning.
A fundamental problem with iterative methods is that they apply spatial smoothing filters throughout the optimization to prevent unphysical deformations. This smoothing is often applied too aggressively, causing oversmoothed deformation fields that falsely deform rigid structures like bones even though those structures barely move during respiration.
Deep learning offers the potential for much faster registration through direct single-step prediction. However, most published deep learning registration methods focus on brain MRI or cardiac images, which have very different properties from abdominal 4D-CT. Brain images have sharp boundaries and small, predictable deformations; abdominal images have large motion, homogeneous tissue regions like liver, and complex unpredictable organ interactions.
Supervised deep learning methods require training data with known ground truth deformations, which are extremely difficult to obtain for abdominal 4D-CT without extensive manual labeling. Artificially generated training deformations are unrealistic and can bias the model. This motivates development of unsupervised approaches that learn from image similarity alone.
The proposed MS-DIRNet (Multi-Scale Deformable Image Registration Network) addresses abdominal motion registration using a two-stage architecture: a GlobalNet that captures broad, large-scale motion from downsampled whole-image volumes, followed by a LocalNet that refines registration accuracy using smaller image patches at full resolution.
The GlobalNet processes downsampled versions of the two CT images (at one-quarter resolution) to predict a global deformation that corrects the large overall organ displacements caused by breathing. The deformed image from GlobalNet is then passed to the LocalNet, which works on overlapping 3D patches to predict the remaining fine-scale local deformations. The final deformation field is the sum of the global and local components.
Both networks share a generative adversarial network (GAN) structure. The generator predicts the deformation field; the discriminator is trained to tell the difference between the deformed image and the target image. When the discriminator can no longer distinguish between the two, the deformed image looks physically realistic -- providing a powerful constraint that goes beyond simple smoothness penalties.
Critically, the entire system is trained in an unsupervised manner: no ground truth deformation fields are needed. The network learns solely from image similarity between the deformed and target images, making it practical to train from routine clinical 4D-CT data without additional manual annotation effort.
A key innovation in the network design is the integration of self-attention gates into the generator architecture. These are learnable modules that detect and highlight regions with significant structural differences between the two input images -- essentially teaching the network to focus its efforts on areas that actually need to be deformed.
In the abdomen, there is a crucial difference between structures that move substantially with breathing (stomach, liver, bowel) and those that remain essentially fixed (spine, bony pelvis). Traditional registration methods treat all regions equally, which is why they often distort bones. Attention gates allow the network to recognize this distinction automatically.
The study demonstrated the value of attention gates through a visualization experiment: image patches from the spine (minimal motion) and stomach (large motion) were encoded by the network, and principal component analysis was applied to the feature representations. Without attention gates, the spine and stomach patches were indistinguishable in feature space. With attention gates, they formed clearly separated clusters -- confirming that the network had learned to recognize motion magnitude as a meaningful characteristic.
The attention gates operate at multiple scales within the encoder, allowing the network to capture both global motion patterns and fine local deformations. Gradients from background regions are down-weighted during training, preventing irrelevant areas from influencing how the network learns to handle motion. This enables bony structures to remain correctly un-deformed while soft tissue registration is optimized.
MS-DIRNet was trained and tested on 4D-CT data from 25 patients with implanted fiducial markers, using five-fold cross-validation. The accuracy metric was target registration error (TRE) -- the distance between corresponding fiducial marker positions after registration, measured in millimeters.
MS-DIRNet achieved an average TRE of 1.2 mm compared to 2.5 mm for the clinically used Varian Velocity commercial software -- more than a 50% improvement in registration accuracy. The improvement was consistent across all nine respiratory phase pairs tested, with the largest relative gains seen in the most challenging pairs (T0 to T5, which represents the maximum breathing displacement).
Ablation experiments confirmed the contribution of each component. Removing the discriminator degraded accuracy from MAE of 25.4 to 29.8 HU. Removing the attention gates also caused a statistically significant drop in performance. Both components were individually necessary and their combination produced the best results -- with the discriminator being particularly critical for handling the most demanding registration cases involving fiducial marker alignment.
The network generates a complete deformation field for a full 4D-CT volume in approximately 2 minutes -- dramatically faster than iterative methods while achieving superior accuracy. Visual inspection showed that MS-DIRNet correctly maintained the position of bony structures while accurately tracking the liver, bowel, and diaphragm boundaries, all of which failed to align correctly with Velocity.
One remaining challenge is registration near tissue-air interfaces, such as the boundary between bowel and gas pockets. Air has negligible CT intensity and provides no texture information for the registration algorithm to work with. When gas pockets shift position between respiratory phases, large apparent intensity differences appear that are difficult to match.
The study used only 25 patients for training and testing, which is a relatively small dataset for deep learning. The authors note that expanding the training set would likely improve performance, particularly for patients with unusual anatomy or extreme respiratory motion. Standardizing image preprocessing to reduce CT artifacts before training would also help.
A potential future improvement is incorporating biomechanical modeling as an additional regularization constraint. Tissue deformation follows physical laws -- different organs have different stiffness and deform in predictable ways. Encoding these properties into the loss function or network architecture could further constrain the predicted deformation fields to be physiologically realistic, particularly important for the liver and diaphragm.
The clinical significance of this work extends beyond prostate cancer to any radiation treatment involving abdominal targets, including pancreatic, liver, and kidney cancers. Fast, accurate 4D-CT registration is also essential for developing adaptive radiotherapy workflows where treatment plans are modified in real time based on daily imaging.