DeepRecon: Joint 2D Cardiac Segmentation and 3D Volume Reconstruction via A Structure-Specific Generative Method

• URL: http://arxiv.org/abs/2206.07163v1
• Date: Tue, 14 Jun 2022 20:46:11 GMT
• Title: DeepRecon: Joint 2D Cardiac Segmentation and 3D Volume Reconstruction via A Structure-Specific Generative Method
• Authors: Qi Chang, Zhennan Yan, Mu Zhou, Di Liu, Khalid Sawalha, Meng Ye, Qilong Zhangli, Mikael Kanski, Subhi Al Aref, Leon Axel, Dimitris Metaxas
• Abstract summary: We propose an end-to-end latent-space-based framework, DeepRecon, that generates multiple clinically essential outcomes. Our method identifies the optimal latent representation of the cine image that contains accurate semantic information for cardiac structures. In particular, our model jointly generates synthetic images with accurate semantic information and segmentation of the cardiac structures.
• Score: 12.26150675728958
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Joint 2D cardiac segmentation and 3D volume reconstruction are fundamental to building statistical cardiac anatomy models and understanding functional mechanisms from motion patterns. However, due to the low through-plane resolution of cine MR and high inter-subject variance, accurately segmenting cardiac images and reconstructing the 3D volume are challenging. In this study, we propose an end-to-end latent-space-based framework, DeepRecon, that generates multiple clinically essential outcomes, including accurate image segmentation, synthetic high-resolution 3D image, and 3D reconstructed volume. Our method identifies the optimal latent representation of the cine image that contains accurate semantic information for cardiac structures. In particular, our model jointly generates synthetic images with accurate semantic information and segmentation of the cardiac structures using the optimal latent representation. We further explore downstream applications of 3D shape reconstruction and 4D motion pattern adaptation by the different latent-space manipulation strategies.The simultaneously generated high-resolution images present a high interpretable value to assess the cardiac shape and motion.Experimental results demonstrate the effectiveness of our approach on multiple fronts including 2D segmentation, 3D reconstruction, downstream 4D motion pattern adaption performance.

Related papers

• MinD-3D: Reconstruct High-quality 3D objects in Human Brain [50.534007259536715]
  Recon3DMind is an innovative task aimed at reconstructing 3D visuals from Functional Magnetic Resonance Imaging (fMRI) signals. We present the fMRI-Shape dataset, which includes data from 14 participants and features 360-degree videos of 3D objects. We propose MinD-3D, a novel and effective three-stage framework specifically designed to decode the brain's 3D visual information from fMRI signals.
  arXiv  Detail & Related papers  (2023-12-12T18:21:36Z)
• Unsupervised 3D Pose Estimation with Non-Rigid Structure-from-Motion Modeling [83.76377808476039]
  We propose a new modeling method for human pose deformations and design an accompanying diffusion-based motion prior. Inspired by the field of non-rigid structure-from-motion, we divide the task of reconstructing 3D human skeletons in motion into the estimation of a 3D reference skeleton. A mixed spatial-temporal NRSfMformer is used to simultaneously estimate the 3D reference skeleton and the skeleton deformation of each frame from 2D observations sequence.
  arXiv  Detail & Related papers  (2023-08-18T16:41:57Z)
• DMCVR: Morphology-Guided Diffusion Model for 3D Cardiac Volume Reconstruction [33.59945107137013]
  Current cardiac MRI-based reconstruction technology is 2D with limited through-plane resolution. We propose a morphology-guided diffusion model for 3D cardiac volume reconstruction, DMCVR, that synthesizes high-resolution 2D images and corresponding 3D reconstructed volumes.
  arXiv  Detail & Related papers  (2023-08-18T00:48:30Z)
• Mesh-based 3D Motion Tracking in Cardiac MRI using Deep Learning [11.177851736773823]
  3D motion estimation from cine cardiac magnetic resonance (CMR) images is important for the assessment of cardiac function and diagnosis of cardiovascular diseases. Most of the previous methods focus on estimating pixel-/voxel-wise motion fields in the full image space. In this work, we model the heart as a 3D geometric mesh and propose a novel deep learning-based method that can estimate 3D motion of the heart mesh from 2D short-axis CMR images.
  arXiv  Detail & Related papers  (2022-09-05T15:10:27Z)
• MulViMotion: Shape-aware 3D Myocardial Motion Tracking from Multi-View Cardiac MRI [11.685829837689404]
  We propose a novel multi-view motion estimation network (MulViMotion) to learn a consistent 3D motion field of the heart. We extensively evaluate the proposed method on 2D cine CMR images from 580 subjects of the UK Biobank study for 3D motion tracking of the left ventricular myocardium.
  arXiv  Detail & Related papers  (2022-07-29T18:29:52Z)
• RiCS: A 2D Self-Occlusion Map for Harmonizing Volumetric Objects [68.85305626324694]
  Ray-marching in Camera Space (RiCS) is a new method to represent the self-occlusions of foreground objects in 3D into a 2D self-occlusion map. We show that our representation map not only allows us to enhance the image quality but also to model temporally coherent complex shadow effects.
  arXiv  Detail & Related papers  (2022-05-14T05:35:35Z)
• H4D: Human 4D Modeling by Learning Neural Compositional Representation [75.34798886466311]
  This work presents a novel framework that can effectively learn a compact and compositional representation for dynamic human. A simple yet effective linear motion model is proposed to provide a rough and regularized motion estimation. Experiments demonstrate our method is not only efficacy in recovering dynamic human with accurate motion and detailed geometry, but also amenable to various 4D human related tasks.
  arXiv  Detail & Related papers  (2022-03-02T17:10:49Z)
• 3D Reconstruction of Curvilinear Structures with Stereo Matching DeepConvolutional Neural Networks [52.710012864395246]
  We propose a fully automated pipeline for both detection and matching of curvilinear structures in stereo pairs. We mainly focus on 3D reconstruction of dislocations from stereo pairs of TEM images.
  arXiv  Detail & Related papers  (2021-10-14T23:05:47Z)
• Joint Motion Correction and Super Resolution for Cardiac Segmentation via Latent Optimisation [18.887520377396925]
  We propose a novel latent optimisation framework that jointly performs motion correction and super resolution for cardiac image segmentations. Experiments on two cardiac MR datasets show that the proposed framework achieves high performance, comparable to state-of-the-art super-resolution approaches.
  arXiv  Detail & Related papers  (2021-07-08T15:14:00Z)
• A Deep-Learning Approach For Direct Whole-Heart Mesh Reconstruction [1.8047694351309207]
  We propose a novel deep-learning-based approach that directly predicts whole heart surface meshes from volumetric CT and MR image data. Our method demonstrated promising performance of generating high-resolution and high-quality whole heart reconstructions.
  arXiv  Detail & Related papers  (2021-02-16T00:39:43Z)
• Hierarchical Amortized Training for Memory-efficient High Resolution 3D GAN [52.851990439671475]
  We propose a novel end-to-end GAN architecture that can generate high-resolution 3D images. We achieve this goal by using different configurations between training and inference. Experiments on 3D thorax CT and brain MRI demonstrate that our approach outperforms state of the art in image generation.
  arXiv  Detail & Related papers  (2020-08-05T02:33:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.