RMSim: Controlled Respiratory Motion Simulation on Static Patient Scans

• URL: http://arxiv.org/abs/2301.11422v1
• Date: Thu, 26 Jan 2023 21:20:14 GMT
• Title: RMSim: Controlled Respiratory Motion Simulation on Static Patient Scans
• Authors: Donghoon Lee, Ellen Yorke, Masoud Zarepisheh, Saad Nadeem, Yu-Chi Hu
• Abstract summary: We present a novel 3D Seq2Seq deep learning respiratory motion simulator (RMSim) that learns from 4D-CT images. 10-phase 4D-CTs of 140 internal patients were used to train and test RMSim. We validated our RMSim output with both private and public benchmark datasets.
• Score: 7.575469466607952
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This work aims to generate realistic anatomical deformations from static patient scans. Specifically, we present a method to generate these deformations/augmentations via deep learning driven respiratory motion simulation that provides the ground truth for validating deformable image registration (DIR) algorithms and driving more accurate deep learning based DIR. We present a novel 3D Seq2Seq deep learning respiratory motion simulator (RMSim) that learns from 4D-CT images and predicts future breathing phases given a static CT image. The predicted respiratory patterns, represented by time-varying displacement vector fields (DVFs) at different breathing phases, are modulated through auxiliary inputs of 1D breathing traces so that a larger amplitude in the trace results in more significant predicted deformation. Stacked 3D-ConvLSTMs are used to capture the spatial-temporal respiration patterns. Training loss includes a smoothness loss in the DVF and mean-squared error between the predicted and ground truth phase images. A spatial transformer deforms the static CT with the predicted DVF to generate the predicted phase image. 10-phase 4D-CTs of 140 internal patients were used to train and test RMSim. The trained RMSim was then used to augment a public DIR challenge dataset for training VoxelMorph to show the effectiveness of RMSim-generated deformation augmentation. We validated our RMSim output with both private and public benchmark datasets (healthy and cancer patients). The proposed approach can be used for validating DIR algorithms as well as for patient-specific augmentations to improve deep learning DIR algorithms. The code, pretrained models, and augmented DIR validation datasets will be released at https://github.com/nadeemlab/SeqX2Y.

Related papers

• Self-Supervised Pre-training Tasks for an fMRI Time-series Transformer in Autism Detection [3.665816629105171]
  Autism Spectrum Disorder (ASD) is a neurodevelopmental condition that encompasses a wide variety of symptoms and degrees of impairment. We have developed a transformer-based self-supervised framework that directly analyzes time-series fMRI data without computing functional connectivity. We show that randomly masking entire ROIs gives better model performance than randomly masking time points in the pre-training step.
  arXiv  Detail & Related papers  (2024-09-18T20:29:23Z)
• Resolving Variable Respiratory Motion From Unsorted 4D Computed Tomography [0.6938240959023204]
  Surrogate driven motion models can estimate variable motion across multiple cycles based on CT segments unsorted' from 4DCT. Our method produces a high-quality motion-compensated image together with estimates of the motion, including breath-to-breath variability.
  arXiv  Detail & Related papers  (2024-06-30T11:22:55Z)
• CathFlow: Self-Supervised Segmentation of Catheters in Interventional Ultrasound Using Optical Flow and Transformers [66.15847237150909]
  We introduce a self-supervised deep learning architecture to segment catheters in longitudinal ultrasound images. The network architecture builds upon AiAReSeg, a segmentation transformer built with the Attention in Attention mechanism. We validated our model on a test dataset, consisting of unseen synthetic data and images collected from silicon aorta phantoms.
  arXiv  Detail & Related papers  (2024-03-21T15:13:36Z)
• ArSDM: Colonoscopy Images Synthesis with Adaptive Refinement Semantic Diffusion Models [69.9178140563928]
  Colonoscopy analysis is essential for assisting clinical diagnosis and treatment. The scarcity of annotated data limits the effectiveness and generalization of existing methods. We propose an Adaptive Refinement Semantic Diffusion Model (ArSDM) to generate colonoscopy images that benefit the downstream tasks.
  arXiv  Detail & Related papers  (2023-09-03T07:55:46Z)
• ORRN: An ODE-based Recursive Registration Network for Deformable Respiratory Motion Estimation with Lung 4DCT Images [7.180268723513929]
  Deformable Image Registration (DIR) plays a significant role in quantifying deformation in medical data. Recent Deep Learning methods have shown promising accuracy and speedup for registering a pair of medical images. This paper presents ORRN, an Ordinary Differential Equations (ODE)-based recursion image registration network.
  arXiv  Detail & Related papers  (2023-05-24T03:26:26Z)
• Joint Rigid Motion Correction and Sparse-View CT via Self-Calibrating Neural Field [37.86878619100209]
  NeRF has widely received attention in Sparse-View (SV) CT reconstruction problems as a self-supervised deep learning framework. Existing NeRF-based SVCT methods strictly suppose there is completely no relative motion during the CT acquisition. This work proposes a self-calibrating neural field that recovers the artifacts-free image from the rigid motion-corrupted SV measurement.
  arXiv  Detail & Related papers  (2022-10-23T13:55:07Z)
• Predicting Loose-Fitting Garment Deformations Using Bone-Driven Motion Networks [63.596602299263935]
  We present a learning algorithm that uses bone-driven motion networks to predict the deformation of loose-fitting garment meshes at interactive rates. We show that our method outperforms state-of-the-art methods in terms of prediction accuracy of mesh deformations by about 20% in RMSE and 10% in Hausdorff distance and STED.
  arXiv  Detail & Related papers  (2022-05-03T07:54:39Z)
• Unsupervised Contrastive Learning based Transformer for Lung Nodule Detection [6.693379403133435]
  Early detection of lung nodules with computed tomography (CT) is critical for the longer survival of lung cancer patients and better quality of life. Computer-aided detection/diagnosis (CAD) is proven valuable as a second or concurrent reader in this context. accurate detection of lung nodules remains a challenge for such CAD systems and even radiologists due to variability in size, location, and appearance of lung nodules. Motivated by recent computer vision techniques, here we present a self-supervised region-based 3D transformer model to identify lung nodules.
  arXiv  Detail & Related papers  (2022-04-30T01:19:00Z)
• Pre-training and Fine-tuning Transformers for fMRI Prediction Tasks [69.85819388753579]
  TFF employs a transformer-based architecture and a two-phase training approach. Self-supervised training is applied to a collection of fMRI scans, where the model is trained for the reconstruction of 3D volume data. Results show state-of-the-art performance on a variety of fMRI tasks, including age and gender prediction, as well as schizophrenia recognition.
  arXiv  Detail & Related papers  (2021-12-10T18:04:26Z)
• Dynamic Mode Decomposition in Adaptive Mesh Refinement and Coarsening Simulations [58.720142291102135]
  Dynamic Mode Decomposition (DMD) is a powerful data-driven method used to extract coherent schemes. This paper proposes a strategy to enable DMD to extract from observations with different mesh topologies and dimensions.
  arXiv  Detail & Related papers  (2021-04-28T22:14:25Z)
• Revisiting 3D Context Modeling with Supervised Pre-training for Universal Lesion Detection in CT Slices [48.85784310158493]
  We propose a Modified Pseudo-3D Feature Pyramid Network (MP3D FPN) to efficiently extract 3D context enhanced 2D features for universal lesion detection in CT slices. With the novel pre-training method, the proposed MP3D FPN achieves state-of-the-art detection performance on the DeepLesion dataset. The proposed 3D pre-trained weights can potentially be used to boost the performance of other 3D medical image analysis tasks.
  arXiv  Detail & Related papers  (2020-12-16T07:11:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.