Mesh-based 3D Motion Tracking in Cardiac MRI using Deep Learning
- URL: http://arxiv.org/abs/2209.02004v1
- Date: Mon, 5 Sep 2022 15:10:27 GMT
- Title: Mesh-based 3D Motion Tracking in Cardiac MRI using Deep Learning
- Authors: Qingjie Meng and Wenjia Bai and Tianrui Liu and Declan P O'Regan and
Daniel Rueckert
- Abstract summary: 3D motion estimation from cine cardiac magnetic resonance (CMR) images is important for the assessment of cardiac function and diagnosis of cardiovascular diseases.
Most of the previous methods focus on estimating pixel-/voxel-wise motion fields in the full image space.
In this work, we model the heart as a 3D geometric mesh and propose a novel deep learning-based method that can estimate 3D motion of the heart mesh from 2D short-axis CMR images.
- Score: 11.177851736773823
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: 3D motion estimation from cine cardiac magnetic resonance (CMR) images is
important for the assessment of cardiac function and diagnosis of
cardiovascular diseases. Most of the previous methods focus on estimating
pixel-/voxel-wise motion fields in the full image space, which ignore the fact
that motion estimation is mainly relevant and useful within the object of
interest, e.g., the heart. In this work, we model the heart as a 3D geometric
mesh and propose a novel deep learning-based method that can estimate 3D motion
of the heart mesh from 2D short- and long-axis CMR images. By developing a
differentiable mesh-to-image rasterizer, the method is able to leverage the
anatomical shape information from 2D multi-view CMR images for 3D motion
estimation. The differentiability of the rasterizer enables us to train the
method end-to-end. One advantage of the proposed method is that by tracking the
motion of each vertex, it is able to keep the vertex correspondence of 3D
meshes between time frames, which is important for quantitative assessment of
the cardiac function on the mesh. We evaluate the proposed method on CMR images
acquired from the UK Biobank study. Experimental results show that the proposed
method quantitatively and qualitatively outperforms both conventional and
learning-based cardiac motion tracking methods.
Related papers
- Explicit Differentiable Slicing and Global Deformation for Cardiac Mesh Reconstruction [8.730291904586656]
Mesh reconstruction of the cardiac anatomy from medical images is useful for shape and motion measurements and biophysics simulations.
Traditional voxel-based approaches rely on pre- and post-processing that compromises image fidelity.
We propose a novel explicit differentiable voxelization and slicing (DVS) algorithm that allows gradient backpropagation to a mesh from its slices.
arXiv Detail & Related papers (2024-09-03T17:19:31Z) - CardioSpectrum: Comprehensive Myocardium Motion Analysis with 3D Deep Learning and Geometric Insights [6.415915756409993]
Conventional neural networks have difficulty predicting subtle tangential movements.
We present a comprehensive approach to address this problem.
Our 3D deep learning architecture, based on the ARFlow model, is optimized to handle complex 3D motion analysis tasks.
arXiv Detail & Related papers (2024-07-04T09:57:44Z) - MinD-3D: Reconstruct High-quality 3D objects in Human Brain [50.534007259536715]
Recon3DMind is an innovative task aimed at reconstructing 3D visuals from Functional Magnetic Resonance Imaging (fMRI) signals.
We present the fMRI-Shape dataset, which includes data from 14 participants and features 360-degree videos of 3D objects.
We propose MinD-3D, a novel and effective three-stage framework specifically designed to decode the brain's 3D visual information from fMRI signals.
arXiv Detail & Related papers (2023-12-12T18:21:36Z) - Continuous 3D Myocardial Motion Tracking via Echocardiography [30.19879953016694]
Myocardial motion tracking is an essential clinical tool in the prevention and detection of cardiovascular diseases.
Current techniques suffer from incomplete and inaccurate motion estimation of the myocardium in both spatial and temporal dimensions.
This paper introduces the Neural Cardiac Motion Field (NeuralCMF) to model the 3D structure and the comprehensive 6D forward/backward motion of the heart.
arXiv Detail & Related papers (2023-10-04T13:11:20Z) - DeepMesh: Mesh-based Cardiac Motion Tracking using Deep Learning [13.289561121562057]
3D motion estimation from cine cardiac magnetic resonance (CMR) images is important for the assessment of cardiac function and the diagnosis of cardiovascular diseases.
In this work, we model the heart as a 3D mesh consisting of epi- and endocardial surfaces.
We propose a novel learning framework, DeepMesh, which propagates a template heart mesh to a subject space and estimates the 3D motion of the heart mesh from CMR images for individual subjects.
arXiv Detail & Related papers (2023-09-25T17:24:18Z) - Deep learning network to correct axial and coronal eye motion in 3D OCT
retinal imaging [65.47834983591957]
We propose deep learning based neural networks to correct axial and coronal motion artifacts in OCT based on a single scan.
The experimental result shows that the proposed method can effectively correct motion artifacts and achieve smaller error than other methods.
arXiv Detail & Related papers (2023-05-27T03:55:19Z) - MulViMotion: Shape-aware 3D Myocardial Motion Tracking from Multi-View
Cardiac MRI [11.685829837689404]
We propose a novel multi-view motion estimation network (MulViMotion) to learn a consistent 3D motion field of the heart.
We extensively evaluate the proposed method on 2D cine CMR images from 580 subjects of the UK Biobank study for 3D motion tracking of the left ventricular myocardium.
arXiv Detail & Related papers (2022-07-29T18:29:52Z) - Revisiting 3D Context Modeling with Supervised Pre-training for
Universal Lesion Detection in CT Slices [48.85784310158493]
We propose a Modified Pseudo-3D Feature Pyramid Network (MP3D FPN) to efficiently extract 3D context enhanced 2D features for universal lesion detection in CT slices.
With the novel pre-training method, the proposed MP3D FPN achieves state-of-the-art detection performance on the DeepLesion dataset.
The proposed 3D pre-trained weights can potentially be used to boost the performance of other 3D medical image analysis tasks.
arXiv Detail & Related papers (2020-12-16T07:11:16Z) - Motion Pyramid Networks for Accurate and Efficient Cardiac Motion
Estimation [51.72616167073565]
We propose Motion Pyramid Networks, a novel deep learning-based approach for accurate and efficient cardiac motion estimation.
We predict and fuse a pyramid of motion fields from multiple scales of feature representations to generate a more refined motion field.
We then use a novel cyclic teacher-student training strategy to make the inference end-to-end and further improve the tracking performance.
arXiv Detail & Related papers (2020-06-28T21:03:19Z) - A Deep Learning Approach for Motion Forecasting Using 4D OCT Data [69.62333053044712]
We propose 4D-temporal deep learning for end-to-end motion forecasting and estimation using a stream of OCT volumes.
Our best performing 4D method achieves motion forecasting with an overall average correlation of 97.41%, while also improving motion estimation performance by a factor of 2.5 compared to a previous 3D approach.
arXiv Detail & Related papers (2020-04-21T15:59:53Z) - Spatio-Temporal Deep Learning Methods for Motion Estimation Using 4D OCT
Image Data [63.73263986460191]
Localizing structures and estimating the motion of a specific target region are common problems for navigation during surgical interventions.
We investigate whether using a temporal stream of OCT image volumes can improve deep learning-based motion estimation performance.
Using 4D information for the model input improves performance while maintaining reasonable inference times.
arXiv Detail & Related papers (2020-04-21T15:43:01Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.