Unsupervised Landmark Detection Based Spatiotemporal Motion Estimation
for 4D Dynamic Medical Images
- URL: http://arxiv.org/abs/2109.14805v1
- Date: Thu, 30 Sep 2021 02:06:02 GMT
- Title: Unsupervised Landmark Detection Based Spatiotemporal Motion Estimation
for 4D Dynamic Medical Images
- Authors: Yuyu Guo, Lei Bi, Dongming Wei, Liyun Chen, Zhengbin Zhu, Dagan Feng,
Ruiyan Zhang, Qian Wang and Jinman Kim
- Abstract summary: We provide a novel motion estimation framework of Dense-Sparse-Dense (DSD), which comprises two stages.
In the first stage, we process the raw dense image to extract sparse landmarks to represent the target organ anatomical topology.
In the second stage, we derive the sparse motion displacement from the extracted sparse landmarks of two images of different time points.
- Score: 16.759486905827433
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Motion estimation is a fundamental step in dynamic medical image processing
for the assessment of target organ anatomy and function. However, existing
image-based motion estimation methods, which optimize the motion field by
evaluating the local image similarity, are prone to produce implausible
estimation, especially in the presence of large motion. In this study, we
provide a novel motion estimation framework of Dense-Sparse-Dense (DSD), which
comprises two stages. In the first stage, we process the raw dense image to
extract sparse landmarks to represent the target organ anatomical topology and
discard the redundant information that is unnecessary for motion estimation.
For this purpose, we introduce an unsupervised 3D landmark detection network to
extract spatially sparse but representative landmarks for the target organ
motion estimation. In the second stage, we derive the sparse motion
displacement from the extracted sparse landmarks of two images of different
time points. Then, we present a motion reconstruction network to construct the
motion field by projecting the sparse landmarks displacement back into the
dense image domain. Furthermore, we employ the estimated motion field from our
two-stage DSD framework as initialization and boost the motion estimation
quality in light-weight yet effective iterative optimization. We evaluate our
method on two dynamic medical imaging tasks to model cardiac motion and lung
respiratory motion, respectively. Our method has produced superior motion
estimation accuracy compared to existing comparative methods. Besides, the
extensive experimental results demonstrate that our solution can extract well
representative anatomical landmarks without any requirement of manual
annotation. Our code is publicly available online.
Related papers
- Attention-aware non-rigid image registration for accelerated MR imaging [10.47044784972188]
We introduce an attention-aware deep learning-based framework that can perform non-rigid pairwise registration for fully sampled and accelerated MRI.
We extract local visual representations to build similarity maps between the registered image pairs at multiple resolution levels.
We demonstrate that our model derives reliable and consistent motion fields across different sampling trajectories.
arXiv Detail & Related papers (2024-04-26T14:25:07Z) - DeepMesh: Mesh-based Cardiac Motion Tracking using Deep Learning [13.289561121562057]
3D motion estimation from cine cardiac magnetic resonance (CMR) images is important for the assessment of cardiac function and the diagnosis of cardiovascular diseases.
In this work, we model the heart as a 3D mesh consisting of epi- and endocardial surfaces.
We propose a novel learning framework, DeepMesh, which propagates a template heart mesh to a subject space and estimates the 3D motion of the heart mesh from CMR images for individual subjects.
arXiv Detail & Related papers (2023-09-25T17:24:18Z) - Mesh-based 3D Motion Tracking in Cardiac MRI using Deep Learning [11.177851736773823]
3D motion estimation from cine cardiac magnetic resonance (CMR) images is important for the assessment of cardiac function and diagnosis of cardiovascular diseases.
Most of the previous methods focus on estimating pixel-/voxel-wise motion fields in the full image space.
In this work, we model the heart as a 3D geometric mesh and propose a novel deep learning-based method that can estimate 3D motion of the heart mesh from 2D short-axis CMR images.
arXiv Detail & Related papers (2022-09-05T15:10:27Z) - Neural Computed Tomography [1.7188280334580197]
Motion during acquisition of a set of projections can lead to significant motion artifacts in computed tomography reconstructions.
We propose a novel reconstruction framework, NeuralCT, to generate time-resolved images free from motion artifacts.
arXiv Detail & Related papers (2022-01-17T18:50:58Z) - MoCaNet: Motion Retargeting in-the-wild via Canonicalization Networks [77.56526918859345]
We present a novel framework that brings the 3D motion task from controlled environments to in-the-wild scenarios.
It is capable of body motion from a character in a 2D monocular video to a 3D character without using any motion capture system or 3D reconstruction procedure.
arXiv Detail & Related papers (2021-12-19T07:52:05Z) - Attentive and Contrastive Learning for Joint Depth and Motion Field
Estimation [76.58256020932312]
Estimating the motion of the camera together with the 3D structure of the scene from a monocular vision system is a complex task.
We present a self-supervised learning framework for 3D object motion field estimation from monocular videos.
arXiv Detail & Related papers (2021-10-13T16:45:01Z) - Inertial Measurements for Motion Compensation in Weight-bearing
Cone-beam CT of the Knee [6.7461735822055715]
Involuntary motion during CT scans of the knee causes artifacts in the reconstructed volumes making them unusable for clinical diagnosis.
We propose to attach an inertial measurement unit (IMU) to the leg of the subject in order to measure the motion during the scan and correct for it.
arXiv Detail & Related papers (2020-07-09T09:26:27Z) - Motion Pyramid Networks for Accurate and Efficient Cardiac Motion
Estimation [51.72616167073565]
We propose Motion Pyramid Networks, a novel deep learning-based approach for accurate and efficient cardiac motion estimation.
We predict and fuse a pyramid of motion fields from multiple scales of feature representations to generate a more refined motion field.
We then use a novel cyclic teacher-student training strategy to make the inference end-to-end and further improve the tracking performance.
arXiv Detail & Related papers (2020-06-28T21:03:19Z) - Appearance Learning for Image-based Motion Estimation in Tomography [60.980769164955454]
In tomographic imaging, anatomical structures are reconstructed by applying a pseudo-inverse forward model to acquired signals.
Patient motion corrupts the geometry alignment in the reconstruction process resulting in motion artifacts.
We propose an appearance learning approach recognizing the structures of rigid motion independently from the scanned object.
arXiv Detail & Related papers (2020-06-18T09:49:11Z) - A Deep Learning Approach for Motion Forecasting Using 4D OCT Data [69.62333053044712]
We propose 4D-temporal deep learning for end-to-end motion forecasting and estimation using a stream of OCT volumes.
Our best performing 4D method achieves motion forecasting with an overall average correlation of 97.41%, while also improving motion estimation performance by a factor of 2.5 compared to a previous 3D approach.
arXiv Detail & Related papers (2020-04-21T15:59:53Z) - Spatio-Temporal Deep Learning Methods for Motion Estimation Using 4D OCT
Image Data [63.73263986460191]
Localizing structures and estimating the motion of a specific target region are common problems for navigation during surgical interventions.
We investigate whether using a temporal stream of OCT image volumes can improve deep learning-based motion estimation performance.
Using 4D information for the model input improves performance while maintaining reasonable inference times.
arXiv Detail & Related papers (2020-04-21T15:43:01Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.