An End-to-End Deep Learning Generative Framework for Refinable Shape Matching and Generation

• URL: http://arxiv.org/abs/2403.06317v1
• Date: Sun, 10 Mar 2024 21:33:53 GMT
• Title: An End-to-End Deep Learning Generative Framework for Refinable Shape Matching and Generation
• Authors: Soodeh Kalaie, Andy Bulpitt, Alejandro F. Frangi, and Ali Gooya
• Abstract summary: Generative modelling for shapes is a prerequisite for In-Silico Clinical Trials (ISCTs) We develop a novel unsupervised geometric deep-learning model to establish refinable shape correspondences in a latent space. We extend our proposed base model to a joint shape generative-clustering multi-atlas framework to incorporate further variability.
• Score: 45.820901263103806
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Generative modelling for shapes is a prerequisite for In-Silico Clinical Trials (ISCTs), which aim to cost-effectively validate medical device interventions using synthetic anatomical shapes, often represented as 3D surface meshes. However, constructing AI models to generate shapes closely resembling the real mesh samples is challenging due to variable vertex counts, connectivities, and the lack of dense vertex-wise correspondences across the training data. Employing graph representations for meshes, we develop a novel unsupervised geometric deep-learning model to establish refinable shape correspondences in a latent space, construct a population-derived atlas and generate realistic synthetic shapes. We additionally extend our proposed base model to a joint shape generative-clustering multi-atlas framework to incorporate further variability and preserve more details in the generated shapes. Experimental results using liver and left-ventricular models demonstrate the approach's applicability to computational medicine, highlighting its suitability for ISCTs through a comparative analysis.

Related papers

• Generative 3D Cardiac Shape Modelling for In-Silico Trials [0.0]
  We propose a deep learning method to model and generate synthetic aortic shapes. The network is trained on a dataset of aortic root meshes reconstructed from CT images. By sampling from the learned embedding vectors, we can generate novel shapes that resemble real patient anatomies.
  arXiv  Detail & Related papers  (2024-09-24T12:59:18Z)
• ReshapeIT: Reliable Shape Interaction with Implicit Template for Anatomical Structure Reconstruction [59.971808117043366]
  ReShapeIT represents an anatomical structure with an implicit template field shared within the same category. It ensures the implicit template field generates valid templates by strengthening the constraint of the correspondence between the instance shape and the template shape. A template Interaction Module is introduced to reconstruct unseen shapes by interacting the valid template shapes with the instance-wise latent codes.
  arXiv  Detail & Related papers  (2023-12-11T07:09:32Z)
• Learning Versatile 3D Shape Generation with Improved AR Models [91.87115744375052]
  Auto-regressive (AR) models have achieved impressive results in 2D image generation by modeling joint distributions in the grid space. We propose the Improved Auto-regressive Model (ImAM) for 3D shape generation, which applies discrete representation learning based on a latent vector instead of volumetric grids.
  arXiv  Detail & Related papers  (2023-03-26T12:03:18Z)
• A Generative Shape Compositional Framework to Synthesise Populations of Virtual Chimaeras [52.33206865588584]
  We introduce a generative shape model for complex anatomical structures, learnable from datasets of unpaired datasets. We build virtual chimaeras from databases of whole-heart shape assemblies that each contribute samples for heart substructures. Our approach significantly outperforms a PCA-based shape model (trained with complete data) in terms of generalisability and specificity.
  arXiv  Detail & Related papers  (2022-10-04T13:36:52Z)
• Autoregressive 3D Shape Generation via Canonical Mapping [92.91282602339398]
  transformers have shown remarkable performances in a variety of generative tasks such as image, audio, and text generation. In this paper, we aim to further exploit the power of transformers and employ them for the task of 3D point cloud generation. Our model can be easily extended to multi-modal shape completion as an application for conditional shape generation.
  arXiv  Detail & Related papers  (2022-04-05T03:12:29Z)
• SP-GAN: Sphere-Guided 3D Shape Generation and Manipulation [50.53931728235875]
  We present SP-GAN, a new unsupervised sphere-guided generative model for direct synthesis of 3D shapes in the form of point clouds. Compared with existing models, SP-GAN is able to synthesize diverse and high-quality shapes with fine details.
  arXiv  Detail & Related papers  (2021-08-10T06:49:45Z)
• GLASS: Geometric Latent Augmentation for Shape Spaces [28.533018136138825]
  We use geometrically motivated energies to augment and thus boost a sparse collection of example (training) models. We analyze the Hessian of the as-rigid-as-possible (ARAP) energy to sample from and project to the underlying (local) shape space. We present multiple examples of interesting and meaningful shape variations even when starting from as few as 3-10 training shapes.
  arXiv  Detail & Related papers  (2021-08-06T17:56:23Z)
• Discriminative and Generative Models for Anatomical Shape Analysison Point Clouds with Deep Neural Networks [3.7814216736076434]
  We introduce deep neural networks for the analysis of anatomical shapes that learn a low-dimensional shape representation from the given task. Our framework is modular and consists of several computing blocks that perform fundamental shape processing tasks. We propose a discriminative model for disease classification and age regression, as well as a generative model for the accruate reconstruction of shapes.
  arXiv  Detail & Related papers  (2020-10-02T07:37:40Z)
• Dynamic multi-object Gaussian process models: A framework for data-driven functional modelling of human joints [0.0]
  A principled and robust way to combine shape and pose features has been illusive due to three main issues. We propose a new framework for dynamic multi-object statistical modelling framework for the analysis of human joints. The framework affords an efficient generative dynamic multi-object modelling platform for biological joints.
  arXiv  Detail & Related papers  (2020-01-22T07:57:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.