Transformer-Guided Graph Attention for Direct Cardiac Mesh Reconstruction: A Structural Digital Twin Framework

• URL: http://arxiv.org/abs/2606.13188v1
• Date: Thu, 11 Jun 2026 10:55:55 GMT
• Title: Transformer-Guided Graph Attention for Direct Cardiac Mesh Reconstruction: A Structural Digital Twin Framework
• Abstract summary: We train a single end-to-end network that goes directly from a raw 3D medical image to a smooth, simulation-ready cardiac surface mesh.<n>The core is a 3D Swin Transformer encoder-decoder that extracts volumetric features from CT or MRI volumes, paired with a Graph Attention Network head that iteratively deforms a template mesh to fit the patient's cardiac boundary.<n>Every mesh is produced in a single forward pass -- no Marching Cubes, no smoothing filters, no manual cleanup.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Building patient-specific cardiac models sits at the heart of precision cardiology, yet getting those models into clinical use keeps running into the same wall: mesh generation is slow, messy, and frustrating. The standard workflow -- segmenting the image, running Marching Cubes, and then manually cleaning up the result -- is time-consuming, inconsistent across operators, and demands specialist knowledge most clinical teams do not have. We take a fundamentally different approach. Instead of treating segmentation and mesh generation as two separate problems, we train a single end-to-end network that goes directly from a raw 3D medical image to a smooth, simulation-ready cardiac surface mesh. The core is a 3D Swin Transformer encoder-decoder that extracts volumetric features from CT or MRI volumes, paired with a Graph Attention Network (GAT) head that iteratively deforms a template mesh to fit the patient's cardiac boundary. We tested on the MM-WHS 2017 benchmark using both CT and MRI. Segmentation scores were competitive (Dice of 0.84 on CT, 0.83 on MRI), but the primary focus is mesh quality: mean Chamfer distance of 1.8 mm, with 95th-percentile surface distance below 5 mm. Every mesh is produced in a single forward pass -- no Marching Cubes, no smoothing filters, no manual cleanup. We argue that for cardiac digital twin pipelines, geometric fidelity and topological correctness matter more than pixel-level Dice scores. By removing the post-processing bottleneck, this approach makes patient-specific cardiac simulation substantially more accessible for clinical use.

Related papers

• Improving Cross-Site Whole-Heart Segmentation [6.716055961221493]
  Whole-heart segmentation from CT and MRI is essential for quantitative cardiac image analysis.<n>We propose a modality-routed 3D cardiac segmentation pipeline that combines TotalSegmentator-componentd nnU-Netv2 models with site-characterized, label-preserving appearance augmentation.<n>Results suggest that site-motivated appearance augmentation is a practical strategy for improving cross-site robustness in limited-data whole-heart segmentation.
  arXiv  Detail & Related papers  (2026-08-25T20:03:53Z)
• Radiology Report Generation with Layer-Wise Anatomical Attention [35.18016233072556]
  We introduce a compact image-to-text architecture that generates Findings section of chest X-ray reports.<n>The model combines a frozen Self-Distillation with No Labels v3 (DINOv3) Vision Transformer (ViT) encoder with a Generative Pre-trained Transformer 2 (GPT-2) decoder.
  arXiv  Detail & Related papers  (2025-12-18T18:17:57Z)
• Full-Head Segmentation of MRI with Abnormal Brain Anatomy: Model and Data Release [1.9492386975158917]
  We collected 98 MRIs with volumetric segmentation labels for a diverse set of human subjects.<n>We developed a MultiAxial network consisting of three 2D U-Net that operate independently in sagittal, axial and coronal planes.
  arXiv  Detail & Related papers  (2025-01-30T19:31:13Z)
• Explicit Differentiable Slicing and Global Deformation for Cardiac Mesh Reconstruction [8.730291904586656]
  Mesh reconstruction of the cardiac anatomy from medical images is useful for shape and motion measurements and biophysics simulations. Traditional voxel-based approaches rely on pre- and post-processing that compromises image fidelity. We propose a novel explicit differentiable voxelization and slicing (DVS) algorithm that allows gradient backpropagation to a mesh from its slices.
  arXiv  Detail & Related papers  (2024-09-03T17:19:31Z)
• MTS-Net: Dual-Enhanced Positional Multi-Head Self-Attention for 3D CT Diagnosis of May-Thurner Syndrome [31.032250821769747]
  May-Thurner Syndrome (MTS) is a vascular condition that affects over 20% of the population and significantly increases the risk of iliofemoral deep venous thrombosis.<n> Accurate and early diagnosis of MTS using computed tomography (CT) remains a clinical challenge due to the subtle anatomical compression and variability across patients.<n>We propose MTS-Net, an end-to-end 3D deep learning framework designed to capture spatial-temporal patterns from CT volumes for reliable MTS diagnosis.
  arXiv  Detail & Related papers  (2024-06-07T06:51:09Z)
• Multi-view Hybrid Graph Convolutional Network for Volume-to-mesh Reconstruction in Cardiovascular MRI [43.47826598981827]
  We introduce HybridVNet, a novel architecture for direct image-to-mesh extraction.<n>We show it can efficiently handle surface and volumetric meshes by encoding them as graph structures.<n>Our model combines traditional convolutional networks with variational graph generative models, deep supervision and mesh-specific regularisation.
  arXiv  Detail & Related papers  (2023-11-22T21:51:29Z)
• Automated deep learning segmentation of high-resolution 7 T postmortem MRI for quantitative analysis of structure-pathology correlations in neurodegenerative diseases [33.191270998887326]
  We present a high resolution of 135 postmortem human brain tissue specimens imaged at 0.3 mm$3$ isotropic using a T2w sequence on a 7T whole-body MRI scanner. We show generalizing capabilities across whole brain hemispheres in different specimens, and also on unseen images acquired at 0.28 mm3 and 0.16 mm3 isotropic T2*w FLASH sequence at 7T.
  arXiv  Detail & Related papers  (2023-03-21T23:44:02Z)
• 3D Matting: A Benchmark Study on Soft Segmentation Method for Pulmonary Nodules Applied in Computed Tomography [32.775884701366465]
  We introduce the image matting into the 3D scenes and use the alpha matte, i.e., a soft mask, to describe lesions in a 3D medical image. To address this issue, we conduct a comprehensive study of 3D matting, including both traditional and deep-learning-based methods. We propose the first end-to-end deep 3D matting network and implement a solid 3D medical image matting benchmark.
  arXiv  Detail & Related papers  (2022-10-11T02:40:18Z)
• Self-supervised 3D anatomy segmentation using self-distilled masked image transformer (SMIT) [2.7298989068857487]
  Self-supervised learning has demonstrated success in medical image segmentation using convolutional networks. We show our approach is more accurate and requires fewer fine tuning datasets than other pretext tasks.
  arXiv  Detail & Related papers  (2022-05-20T17:55:14Z)
• A Novel Hybrid Convolutional Neural Network for Accurate Organ Segmentation in 3D Head and Neck CT Images [31.245109574064532]
  We propose a novel hybrid CNN that fuses 2D and 3D convolutions to combat the different spatial resolutions and extract effective edge and semantic features from 3D HaN CT images. Experiments on the MICCAI 2015 challenge dataset demonstrate that OrganNet2.5D achieves promising performance compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-09-26T15:37:47Z)
• Segmentation of Cardiac Structures via Successive Subspace Learning with Saab Transform from Cine MRI [29.894633364282555]
  We propose a machine learning model, successive subspace learning with the subspace approximation with adjusted bias (Saab) transform, for accurate and efficient segmentation from cine MRI. Our framework performed better than state-of-the-art U-Net models with 200$times$ fewer parameters in the left ventricle, right ventricle, and myocardium.
  arXiv  Detail & Related papers  (2021-07-22T14:50:48Z)
• PC-U Net: Learning to Jointly Reconstruct and Segment the Cardiac Walls in 3D from CT Data [18.941064150226236]
  We propose a PC-U net that jointly reconstructs the point cloud of the LV MYO wall directly from volumes of 2D CT slices. The proposed joint learning framework of our PC-U net is beneficial for automatic cardiac image analysis tasks.
  arXiv  Detail & Related papers  (2020-08-18T23:37:05Z)
• CPR-GCN: Conditional Partial-Residual Graph Convolutional Network in Automated Anatomical Labeling of Coronary Arteries [34.47036303404335]
  We propose a conditional partial-residual graph convolutional network (CPR-GCN), which takes both position and CT image into consideration. According to the five-fold cross-validation, our CPR-GCN yields 95.8% meanRecall, 95.4% meanPrecision and 0.955 meanF1, which outperforms state-of-the-art approaches.
  arXiv  Detail & Related papers  (2020-03-19T04:02:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.