Unsupervised Landmark Detection Based Spatiotemporal Motion Estimation for 4D Dynamic Medical Images

• URL: http://arxiv.org/abs/2109.14805v1
• Date: Thu, 30 Sep 2021 02:06:02 GMT
• Title: Unsupervised Landmark Detection Based Spatiotemporal Motion Estimation for 4D Dynamic Medical Images
• Authors: Yuyu Guo, Lei Bi, Dongming Wei, Liyun Chen, Zhengbin Zhu, Dagan Feng, Ruiyan Zhang, Qian Wang and Jinman Kim
• Abstract summary: We provide a novel motion estimation framework of Dense-Sparse-Dense (DSD), which comprises two stages. In the first stage, we process the raw dense image to extract sparse landmarks to represent the target organ anatomical topology. In the second stage, we derive the sparse motion displacement from the extracted sparse landmarks of two images of different time points.
• Score: 16.759486905827433
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Motion estimation is a fundamental step in dynamic medical image processing for the assessment of target organ anatomy and function. However, existing image-based motion estimation methods, which optimize the motion field by evaluating the local image similarity, are prone to produce implausible estimation, especially in the presence of large motion. In this study, we provide a novel motion estimation framework of Dense-Sparse-Dense (DSD), which comprises two stages. In the first stage, we process the raw dense image to extract sparse landmarks to represent the target organ anatomical topology and discard the redundant information that is unnecessary for motion estimation. For this purpose, we introduce an unsupervised 3D landmark detection network to extract spatially sparse but representative landmarks for the target organ motion estimation. In the second stage, we derive the sparse motion displacement from the extracted sparse landmarks of two images of different time points. Then, we present a motion reconstruction network to construct the motion field by projecting the sparse landmarks displacement back into the dense image domain. Furthermore, we employ the estimated motion field from our two-stage DSD framework as initialization and boost the motion estimation quality in light-weight yet effective iterative optimization. We evaluate our method on two dynamic medical imaging tasks to model cardiac motion and lung respiratory motion, respectively. Our method has produced superior motion estimation accuracy compared to existing comparative methods. Besides, the extensive experimental results demonstrate that our solution can extract well representative anatomical landmarks without any requirement of manual annotation. Our code is publicly available online.

Related papers

• Highly efficient non-rigid registration in k-space with application to cardiac Magnetic Resonance Imaging [10.618048010632728]
  We propose a novel self-supervised deep learning-based framework, dubbed the Local-All Pass Attention Network (LAPANet) for non-rigid motion estimation. LAPANet was evaluated on cardiac motion estimation across various sampling trajectories and acceleration rates. The achieved high temporal resolution (less than 5 ms) for non-rigid motion opens new avenues for motion detection, tracking and correction in dynamic and real-time MRI applications.
  arXiv  Detail & Related papers  (2024-10-24T15:19:59Z)
• MotionTTT: 2D Test-Time-Training Motion Estimation for 3D Motion Corrected MRI [24.048132427816704]
  We propose a deep learning-based test-time-training method for accurate motion estimation. We show that our method can provably reconstruct motion parameters for a simple signal and neural network model.
  arXiv  Detail & Related papers  (2024-09-14T08:51:33Z)
• Attention-aware non-rigid image registration for accelerated MR imaging [10.47044784972188]
  We introduce an attention-aware deep learning-based framework that can perform non-rigid pairwise registration for fully sampled and accelerated MRI. We extract local visual representations to build similarity maps between the registered image pairs at multiple resolution levels. We demonstrate that our model derives reliable and consistent motion fields across different sampling trajectories.
  arXiv  Detail & Related papers  (2024-04-26T14:25:07Z)
• Mesh-based 3D Motion Tracking in Cardiac MRI using Deep Learning [11.177851736773823]
  3D motion estimation from cine cardiac magnetic resonance (CMR) images is important for the assessment of cardiac function and diagnosis of cardiovascular diseases. Most of the previous methods focus on estimating pixel-/voxel-wise motion fields in the full image space. In this work, we model the heart as a 3D geometric mesh and propose a novel deep learning-based method that can estimate 3D motion of the heart mesh from 2D short-axis CMR images.
  arXiv  Detail & Related papers  (2022-09-05T15:10:27Z)
• MoCaNet: Motion Retargeting in-the-wild via Canonicalization Networks [77.56526918859345]
  We present a novel framework that brings the 3D motion task from controlled environments to in-the-wild scenarios. It is capable of body motion from a character in a 2D monocular video to a 3D character without using any motion capture system or 3D reconstruction procedure.
  arXiv  Detail & Related papers  (2021-12-19T07:52:05Z)
• Attentive and Contrastive Learning for Joint Depth and Motion Field Estimation [76.58256020932312]
  Estimating the motion of the camera together with the 3D structure of the scene from a monocular vision system is a complex task. We present a self-supervised learning framework for 3D object motion field estimation from monocular videos.
  arXiv  Detail & Related papers  (2021-10-13T16:45:01Z)
• Inertial Measurements for Motion Compensation in Weight-bearing Cone-beam CT of the Knee [6.7461735822055715]
  Involuntary motion during CT scans of the knee causes artifacts in the reconstructed volumes making them unusable for clinical diagnosis. We propose to attach an inertial measurement unit (IMU) to the leg of the subject in order to measure the motion during the scan and correct for it.
  arXiv  Detail & Related papers  (2020-07-09T09:26:27Z)
• Motion Pyramid Networks for Accurate and Efficient Cardiac Motion Estimation [51.72616167073565]
  We propose Motion Pyramid Networks, a novel deep learning-based approach for accurate and efficient cardiac motion estimation. We predict and fuse a pyramid of motion fields from multiple scales of feature representations to generate a more refined motion field. We then use a novel cyclic teacher-student training strategy to make the inference end-to-end and further improve the tracking performance.
  arXiv  Detail & Related papers  (2020-06-28T21:03:19Z)
• Appearance Learning for Image-based Motion Estimation in Tomography [60.980769164955454]
  In tomographic imaging, anatomical structures are reconstructed by applying a pseudo-inverse forward model to acquired signals. Patient motion corrupts the geometry alignment in the reconstruction process resulting in motion artifacts. We propose an appearance learning approach recognizing the structures of rigid motion independently from the scanned object.
  arXiv  Detail & Related papers  (2020-06-18T09:49:11Z)
• A Deep Learning Approach for Motion Forecasting Using 4D OCT Data [69.62333053044712]
  We propose 4D-temporal deep learning for end-to-end motion forecasting and estimation using a stream of OCT volumes. Our best performing 4D method achieves motion forecasting with an overall average correlation of 97.41%, while also improving motion estimation performance by a factor of 2.5 compared to a previous 3D approach.
  arXiv  Detail & Related papers  (2020-04-21T15:59:53Z)
• Spatio-Temporal Deep Learning Methods for Motion Estimation Using 4D OCT Image Data [63.73263986460191]
  Localizing structures and estimating the motion of a specific target region are common problems for navigation during surgical interventions. We investigate whether using a temporal stream of OCT image volumes can improve deep learning-based motion estimation performance. Using 4D information for the model input improves performance while maintaining reasonable inference times.
  arXiv  Detail & Related papers  (2020-04-21T15:43:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.