SpaER: Learning Spatio-temporal Equivariant Representations for Fetal Brain Motion Tracking

• URL: http://arxiv.org/abs/2407.20198v3
• Date: Wed, 31 Jul 2024 15:36:22 GMT
• Title: SpaER: Learning Spatio-temporal Equivariant Representations for Fetal Brain Motion Tracking
• Authors: Jian Wang, Razieh Faghihpirayesh, Polina Golland, Ali Gholipour,
• Abstract summary: SpaER is a pioneering method for fetal motion tracking. We develop an equivariant neural network that efficiently learns rigid motion sequences. We validate our model using real fetal echo-planar images with simulated and real motions.
• Score: 6.417960463128722
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we introduce SpaER, a pioneering method for fetal motion tracking that leverages equivariant filters and self-attention mechanisms to effectively learn spatio-temporal representations. Different from conventional approaches that statically estimate fetal brain motions from pairs of images, our method dynamically tracks the rigid movement patterns of the fetal head across temporal and spatial dimensions. Specifically, we first develop an equivariant neural network that efficiently learns rigid motion sequences through low-dimensional spatial representations of images. Subsequently, we learn spatio-temporal representations by incorporating time encoding and self-attention neural network layers. This approach allows for the capture of long-term dependencies of fetal brain motion and addresses alignment errors due to contrast changes and severe motion artifacts. Our model also provides a geometric deformation estimation that properly addresses image distortions among all time frames. To the best of our knowledge, our approach is the first to learn spatial-temporal representations via deep neural networks for fetal motion tracking without data augmentation. We validated our model using real fetal echo-planar images with simulated and real motions. Our method carries significant potential value in accurately measuring, tracking, and correcting fetal motion in fetal MRI sequences.

Related papers

• Highly efficient non-rigid registration in k-space with application to cardiac Magnetic Resonance Imaging [10.618048010632728]
  We propose a novel self-supervised deep learning-based framework, dubbed the Local-All Pass Attention Network (LAPANet) for non-rigid motion estimation. LAPANet was evaluated on cardiac motion estimation across various sampling trajectories and acceleration rates. The achieved high temporal resolution (less than 5 ms) for non-rigid motion opens new avenues for motion detection, tracking and correction in dynamic and real-time MRI applications.
  arXiv  Detail & Related papers  (2024-10-24T15:19:59Z)
• Fully Unsupervised Dynamic MRI Reconstruction via Diffeo-Temporal Equivariance [2.260147251787331]
  Supervised learning methods are flawed as they assume periodicity, disallowing imaging of true motion. We propose an unsupervised framework to learn to reconstruct dynamic MRI sequences from undersampled measurements alone. Our method is agnostic to the underlying neural network architecture and can be used to adapt the latest paradigms and post-processing approaches.
  arXiv  Detail & Related papers  (2024-10-11T09:16:30Z)
• Intraoperative Registration by Cross-Modal Inverse Neural Rendering [61.687068931599846]
  We present a novel approach for 3D/2D intraoperative registration during neurosurgery via cross-modal inverse neural rendering. Our approach separates implicit neural representation into two components, handling anatomical structure preoperatively and appearance intraoperatively. We tested our method on retrospective patients' data from clinical cases, showing that our method outperforms state-of-the-art while meeting current clinical standards for registration.
  arXiv  Detail & Related papers  (2024-09-18T13:40:59Z)
• Gear-NeRF: Free-Viewpoint Rendering and Tracking with Motion-aware Spatio-Temporal Sampling [70.34875558830241]
  We present a way for learning a-temporal (4D) embedding, based on semantic semantic gears to allow for stratified modeling of dynamic regions of rendering the scene. At the same time, almost for free, our tracking approach enables free-viewpoint of interest - a functionality not yet achieved by existing NeRF-based methods.
  arXiv  Detail & Related papers  (2024-06-06T03:37:39Z)
• Equivariant Graph Neural Operator for Modeling 3D Dynamics [148.98826858078556]
  We propose Equivariant Graph Neural Operator (EGNO) to directly models dynamics as trajectories instead of just next-step prediction. EGNO explicitly learns the temporal evolution of 3D dynamics where we formulate the dynamics as a function over time and learn neural operators to approximate it. Comprehensive experiments in multiple domains, including particle simulations, human motion capture, and molecular dynamics, demonstrate the significantly superior performance of EGNO against existing methods.
  arXiv  Detail & Related papers  (2024-01-19T21:50:32Z)
• Spatio-Temporal Branching for Motion Prediction using Motion Increments [55.68088298632865]
  Human motion prediction (HMP) has emerged as a popular research topic due to its diverse applications. Traditional methods rely on hand-crafted features and machine learning techniques. We propose a noveltemporal-temporal branching network using incremental information for HMP.
  arXiv  Detail & Related papers  (2023-08-02T12:04:28Z)
• DRIMET: Deep Registration for 3D Incompressible Motion Estimation in Tagged-MRI with Application to the Tongue [11.485843032637439]
  Tagged magnetic resonance imaging(MRI) has been used for decades to observe and quantify the detailed motion of deforming tissue. This paper presents a novel unsupervised phase-based 3D motion estimation technique for tagged MRI.
  arXiv  Detail & Related papers  (2023-01-18T00:16:30Z)
• Unsupervised inter-frame motion correction for whole-body dynamic PET using convolutional long short-term memory in a convolutional neural network [9.349668170221975]
  We develop an unsupervised deep learning-based framework to correct inter-frame body motion. The motion estimation network is a convolutional neural network with a combined convolutional long short-term memory layer. Once trained, the motion estimation inference time of our proposed network was around 460 times faster than the conventional registration baseline.
  arXiv  Detail & Related papers  (2022-06-13T17:38:16Z)
• Neural Computed Tomography [1.7188280334580197]
  Motion during acquisition of a set of projections can lead to significant motion artifacts in computed tomography reconstructions. We propose a novel reconstruction framework, NeuralCT, to generate time-resolved images free from motion artifacts.
  arXiv  Detail & Related papers  (2022-01-17T18:50:58Z)
• 4D Spatio-Temporal Convolutional Networks for Object Position Estimation in OCT Volumes [69.62333053044712]
  3D convolutional neural networks (CNNs) have shown promising performance for pose estimation of a marker object using single OCT images. We extend 3D CNNs to 4D-temporal CNNs to evaluate the impact of additional temporal information for marker object tracking.
  arXiv  Detail & Related papers  (2020-07-02T12:02:20Z)
• Spatio-Temporal Deep Learning Methods for Motion Estimation Using 4D OCT Image Data [63.73263986460191]
  Localizing structures and estimating the motion of a specific target region are common problems for navigation during surgical interventions. We investigate whether using a temporal stream of OCT image volumes can improve deep learning-based motion estimation performance. Using 4D information for the model input improves performance while maintaining reasonable inference times.
  arXiv  Detail & Related papers  (2020-04-21T15:43:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.