NISF++: Geometrically-grounded implicit representations of 3D+time cardiac function from 2D short- and long-axis MR views

• URL: http://arxiv.org/abs/2608.00752v1
• Date: Sat, 01 Aug 2026 16:35:56 GMT
• Title: NISF++: Geometrically-grounded implicit representations of 3D+time cardiac function from 2D short- and long-axis MR views
• Abstract summary: This paper builds upon previous work on neural implicit segmentation functions (NISF) to overcome unaddressed challenges in cardiac function modeling.<n>Our architecture builds a shared 3D+time representations from all available acquisition planes regardless of orientation.<n>Our in-plane segmentation performance is on-par with existing CMR segmentation methods and explore how the majority of failure cases arise from limitations in the ground-truth segmentation.
• Score: 17.503826895619614
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Clinical acquisition in cardiac magnetic resonance (CMR) imaging involves obtaining cross-sectional planes of the heart along the radial and longitudinal directions. Despite these planes being 2D cross-sectional images of the heart, radiologists understand the 3D spatial and continuous temporal nature of the organ being imaged. The same can not be said about the conventional deep learning architectures used to process CMR images, which rely on in-plane and grid-based operations, and are hence unable to organically integrate information from all imaging planes. This paper builds upon previous work on neural implicit segmentation functions (NISF) to overcome unaddressed challenges in cardiac function modeling in the CMR domain. For a given subject, our architecture builds a shared 3D+time representations from all available acquisition planes regardless of orientation. By design, predictions along any imaging plane orientation are cross-sections of the same 3D representation, leading to spatio-temporal consistency across all slices. Moreover, our architecture makes the rotation and translation parameters of imaging planes learnable, allowing us to correct for the commonplace respiratory and patient motion between slice acquisitions under a rigid assumption. Furthermore, interpolation of intensities and segmentation can be performed in 4D at any desired resolution. We perform our study on a 120 subject sub-cohort of CMR imaging data from the UK-Biobank. Our in-plane segmentation performance is on-par with existing CMR segmentation methods and explore how the majority of failure cases arise from limitations in the ground-truth segmentation, for which our representations make predictions with better anatomical accuracy than its original training data. We also evaluate our motion-correction capabilities, displaying quantitative and qualitative improvements in slice alignment.

Related papers

• Axial-Centric Cross-Plane Attention for 3D Medical Image Classification [0.14868610461685575]
  Clinicians commonly interpret 3D medical images using multiple anatomical planes rather than as a single representation.<n>We propose an axial-centric cross-plane attention architecture for 3D medical image classification.<n>Our architecture incorporates MedDINOv3, a medical vision foundation model pretrained via self-supervised learning on large-scale axial CT images.
  arXiv  Detail & Related papers  (2026-02-25T07:00:24Z)
• Dyna3DGR: 4D Cardiac Motion Tracking with Dynamic 3D Gaussian Representation [23.163504377816043]
  We propose Dynamic 3D Gaussian Representation (Dyna3DGR), a novel framework that combines explicit 3D Gaussian representation with implicit neural motion field modeling.<n>Our method simultaneously optimize cardiac structure and motion in a self-supervised manner, eliminating the need for extensive training data or point-to-point correspondences.
  arXiv  Detail & Related papers  (2025-07-22T14:06:50Z)
• MedTet: An Online Motion Model for 4D Heart Reconstruction [59.74234226055964]
  We present a novel approach to reconstruction of 3D cardiac motion from sparse intraoperative data.<n>Existing methods can accurately reconstruct 3D organ geometries from full 3D volumetric imaging.<n>We propose a versatile framework for reconstructing 3D motion from such partial data.
  arXiv  Detail & Related papers  (2024-12-03T17:18:33Z)
• Whole Heart 3D+T Representation Learning Through Sparse 2D Cardiac MR Images [13.686473040836113]
  We introduce a whole-heart self-supervised learning framework to automatically uncover the correlations between spatial and temporal patches throughout the cardiac stacks. We train our model on 14,000 unlabeled CMR data from UK BioBank and evaluate it on 1,000 annotated data.
  arXiv  Detail & Related papers  (2024-06-01T07:08:45Z)
• CMRxRecon: An open cardiac MRI dataset for the competition of accelerated image reconstruction [62.61209705638161]
  There has been growing interest in deep learning-based CMR imaging algorithms. Deep learning methods require large training datasets. This dataset includes multi-contrast, multi-view, multi-slice and multi-coil CMR imaging data from 300 subjects.
  arXiv  Detail & Related papers  (2023-09-19T15:14:42Z)
• Perspective Projection-Based 3D CT Reconstruction from Biplanar X-rays [32.98966469644061]
  We propose PerX2CT, a novel CT reconstruction framework from X-ray. Our proposed method provides a different combination of features for each coordinate which implicitly allows the model to obtain information about the 3D location.
  arXiv  Detail & Related papers  (2023-03-09T14:45:25Z)
• Attentive Symmetric Autoencoder for Brain MRI Segmentation [56.02577247523737]
  We propose a novel Attentive Symmetric Auto-encoder based on Vision Transformer (ViT) for 3D brain MRI segmentation tasks. In the pre-training stage, the proposed auto-encoder pays more attention to reconstruct the informative patches according to the gradient metrics. Experimental results show that our proposed attentive symmetric auto-encoder outperforms the state-of-the-art self-supervised learning methods and medical image segmentation models.
  arXiv  Detail & Related papers  (2022-09-19T09:43:19Z)
• DeepRecon: Joint 2D Cardiac Segmentation and 3D Volume Reconstruction via A Structure-Specific Generative Method [12.26150675728958]
  We propose an end-to-end latent-space-based framework, DeepRecon, that generates multiple clinically essential outcomes. Our method identifies the optimal latent representation of the cine image that contains accurate semantic information for cardiac structures. In particular, our model jointly generates synthetic images with accurate semantic information and segmentation of the cardiac structures.
  arXiv  Detail & Related papers  (2022-06-14T20:46:11Z)
• Two-Stream Graph Convolutional Network for Intra-oral Scanner Image Segmentation [133.02190910009384]
  We propose a two-stream graph convolutional network (i.e., TSGCN) to handle inter-view confusion between different raw attributes. Our TSGCN significantly outperforms state-of-the-art methods in 3D tooth (surface) segmentation.
  arXiv  Detail & Related papers  (2022-04-19T10:41:09Z)
• A unified 3D framework for Organs at Risk Localization and Segmentation for Radiation Therapy Planning [56.52933974838905]
  Current medical workflow requires manual delineation of organs-at-risk (OAR) In this work, we aim to introduce a unified 3D pipeline for OAR localization-segmentation. Our proposed framework fully enables the exploitation of 3D context information inherent in medical imaging.
  arXiv  Detail & Related papers  (2022-03-01T17:08:41Z)
• 3D Reconstruction of Curvilinear Structures with Stereo Matching DeepConvolutional Neural Networks [52.710012864395246]
  We propose a fully automated pipeline for both detection and matching of curvilinear structures in stereo pairs. We mainly focus on 3D reconstruction of dislocations from stereo pairs of TEM images.
  arXiv  Detail & Related papers  (2021-10-14T23:05:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.