UltraTwin: Towards Cardiac Anatomical Twin Generation from Multi-view 2D Ultrasound

• URL: http://arxiv.org/abs/2506.23490v1
• Date: Mon, 30 Jun 2025 03:27:42 GMT
• Title: UltraTwin: Towards Cardiac Anatomical Twin Generation from Multi-view 2D Ultrasound
• Authors: Junxuan Yu, Yaofei Duan, Yuhao Huang, Yu Wang, Rongbo Ling, Weihao Luo, Ang Zhang, Jingxian Xu, Qiongying Ni, Yongsong Zhou, Binghan Li, Haoran Dou, Liping Liu, Yanfen Chu, Feng Geng, Zhe Sheng, Zhifeng Ding, Dingxin Zhang, Rui Huang, Yuhang Zhang, Xiaowei Xu, Tao Tan, Dong Ni, Zhongshan Gou, Xin Yang,
• Abstract summary: We introduce a novel generative framework UltraTwin, to obtain cardiac anatomical twin from sparse multi-view 2D US.<n>First, pioneered the construction of a real-world and high-quality dataset containing strictly paired multi-view 2D US and CT.<n>Second, we propose a coarse-to-fine scheme to achieve hierarchical reconstruction optimization.<n>Third, we introduce an implicit autoencoder for topology-aware constraints.
• Score: 17.70782591089384
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Echocardiography is routine for cardiac examination. However, 2D ultrasound (US) struggles with accurate metric calculation and direct observation of 3D cardiac structures. Moreover, 3D US is limited by low resolution, small field of view and scarce availability in practice. Constructing the cardiac anatomical twin from 2D images is promising to provide precise treatment planning and clinical quantification. However, it remains challenging due to the rare paired data, complex structures, and US noises. In this study, we introduce a novel generative framework UltraTwin, to obtain cardiac anatomical twin from sparse multi-view 2D US. Our contribution is three-fold. First, pioneered the construction of a real-world and high-quality dataset containing strictly paired multi-view 2D US and CT, and pseudo-paired data. Second, we propose a coarse-to-fine scheme to achieve hierarchical reconstruction optimization. Last, we introduce an implicit autoencoder for topology-aware constraints. Extensive experiments show that UltraTwin reconstructs high-quality anatomical twins versus strong competitors. We believe it advances anatomical twin modeling for potential applications in personalized cardiac care.

Related papers

• 3D Heart Reconstruction from Sparse Pose-agnostic 2D Echocardiographic Slices [6.275809518974307]
  We propose an innovative framework for reconstructing personalized 3D heart anatomy from 2D echo slices.<n>When six planes are used, the reconstructed 3D heart can lead to a significant improvement for LV volume estimation.<n>This study provides a new way for personalized 3D structure and function analysis from cardiac ultrasound.
  arXiv  Detail & Related papers  (2025-07-03T08:13:07Z)
• S2MNet: Speckle-To-Mesh Net for Three-Dimensional Cardiac Morphology Reconstruction via Echocardiogram [8.28431711854234]
  Most clinical echocardiograms provide only two-dimensional views, limiting the ability to fully assess cardiac anatomy and function in three dimensions.<n>We propose a deep learning framework S2MNet that reconstructs continuous and high-fidelity 3D heart models by integrating six slices of routinely acquired 2D echocardiogram views.
  arXiv  Detail & Related papers  (2025-05-09T14:56:48Z)
• MedTet: An Online Motion Model for 4D Heart Reconstruction [59.74234226055964]
  We present a novel approach to reconstruction of 3D cardiac motion from sparse intraoperative data.<n>Existing methods can accurately reconstruct 3D organ geometries from full 3D volumetric imaging.<n>We propose a versatile framework for reconstructing 3D motion from such partial data.
  arXiv  Detail & Related papers  (2024-12-03T17:18:33Z)
• NeCA: 3D Coronary Artery Tree Reconstruction from Two 2D Projections via Neural Implicit Representation [2.1771042711033997]
  2D X-ray invasive coronary angiography remains the most widely adopted imaging modality for CVD assessment during real-time cardiac interventions.<n>Due to the radiation limit, often only two angiographic projections are acquired, providing limited information of the vessel geometry.<n>We propose a self-supervised deep learning method called NeCA, which is based on neural implicit representation using the multiresolution hash encoder and differentiable cone-beam forward projector layer.
  arXiv  Detail & Related papers  (2024-09-06T20:08:21Z)
• Sequence-aware Pre-training for Echocardiography Probe Guidance [66.35766658717205]
  Cardiac ultrasound faces two major challenges: (1) the inherently complex structure of the heart, and (2) significant individual variations. Previous works have only learned the population-averaged 2D and 3D structures of the heart rather than personalized cardiac structural features. We propose a sequence-aware self-supervised pre-training method to learn personalized 2D and 3D cardiac structural features.
  arXiv  Detail & Related papers  (2024-08-27T12:55:54Z)
• Structure-aware World Model for Probe Guidance via Large-scale Self-supervised Pre-train [66.35766658717205]
  Successful echocardiography requires a thorough understanding of the structures on the two-dimensional plane and the spatial relationships between planes in three-dimensional space. We propose a large-scale self-supervised pre-training method to acquire a cardiac structure-aware world model.
  arXiv  Detail & Related papers  (2024-06-28T08:54:44Z)
• Epicardium Prompt-guided Real-time Cardiac Ultrasound Frame-to-volume Registration [50.602074919305636]
  This paper introduces a lightweight end-to-end Cardiac Ultrasound frame-to-volume Registration network, termed CU-Reg.<n>We use epicardium prompt-guided anatomical clues to reinforce the interaction of 2D sparse and 3D dense features, followed by a voxel-wise local-global aggregation of enhanced features.
  arXiv  Detail & Related papers  (2024-06-20T17:47:30Z)
• Cardiac Copilot: Automatic Probe Guidance for Echocardiography with World Model [66.35766658717205]
  There is a severe shortage of experienced cardiac sonographers, due to the heart's complex structure and significant operational challenges. We present a Cardiac Copilot system capable of providing real-time probe movement guidance. The core innovation lies in proposing a data-driven world model, named Cardiac Dreamer, for representing cardiac spatial structures. We train our model with real-world ultrasound data and corresponding probe motion from 110 routine clinical scans with 151K sample pairs by three certified sonographers.
  arXiv  Detail & Related papers  (2024-06-19T02:42:29Z)
• Multi-class point cloud completion networks for 3D cardiac anatomy reconstruction from cine magnetic resonance images [4.1448595037512925]
  We propose a novel fully automatic surface reconstruction pipeline capable of reconstructing multi-class 3D cardiac anatomy meshes. Its key component is a multi-class point cloud completion network (PCCN) capable of correcting both the sparsity and misalignment issues of the 3D reconstruction task.
  arXiv  Detail & Related papers  (2023-07-17T14:52:52Z)
• Efficient Pix2Vox++ for 3D Cardiac Reconstruction from 2D echo views [0.6524460254566904]
  Reconstructing cardiac anatomy in 3D can enable discovery of new biomarkers. Most ultrasound systems only have 2D imaging capabilities. We propose a pipeline to perform reconstruction of 3D anatomy from 2D standard cardiac views.
  arXiv  Detail & Related papers  (2022-07-27T10:05:46Z)
• XraySyn: Realistic View Synthesis From a Single Radiograph Through CT Priors [118.27130593216096]
  A radiograph visualizes the internal anatomy of a patient through the use of X-ray, which projects 3D information onto a 2D plane. To the best of our knowledge, this is the first work on radiograph view synthesis. We show that by gaining an understanding of radiography in 3D space, our method can be applied to radiograph bone extraction and suppression without groundtruth bone labels.
  arXiv  Detail & Related papers  (2020-12-04T05:08:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.