LaB-GATr: geometric algebra transformers for large biomedical surface and volume meshes

• URL: http://arxiv.org/abs/2403.07536v2
• Date: Sun, 03 Nov 2024 19:21:04 GMT
• Title: LaB-GATr: geometric algebra transformers for large biomedical surface and volume meshes
• Authors: Julian Suk, Baris Imre, Jelmer M. Wolterink,
• Abstract summary: We propose LaB-GATr, a deep neural network with geometric tokenisation for learning with high-fidelity meshes. LaB-GATr achieves state-of-the-art results on three tasks in cardiovascular hemodynamics modelling and neurodevelopmental phenotype prediction. Our results demonstrate that LaB-GATr is a powerful architecture for learning with high-fidelity meshes which has the potential to enable interesting downstream applications.
• Score: 1.4637995279014533
• License:
• Abstract: Many anatomical structures can be described by surface or volume meshes. Machine learning is a promising tool to extract information from these 3D models. However, high-fidelity meshes often contain hundreds of thousands of vertices, which creates unique challenges in building deep neural network architectures. Furthermore, patient-specific meshes may not be canonically aligned which limits the generalisation of machine learning algorithms. We propose LaB-GATr, a transfomer neural network with geometric tokenisation that can effectively learn with large-scale (bio-)medical surface and volume meshes through sequence compression and interpolation. Our method extends the recently proposed geometric algebra transformer (GATr) and thus respects all Euclidean symmetries, i.e. rotation, translation and reflection, effectively mitigating the problem of canonical alignment between patients. LaB-GATr achieves state-of-the-art results on three tasks in cardiovascular hemodynamics modelling and neurodevelopmental phenotype prediction, featuring meshes of up to 200,000 vertices. Our results demonstrate that LaB-GATr is a powerful architecture for learning with high-fidelity meshes which has the potential to enable interesting downstream applications. Our implementation is publicly available.

Related papers

• NASM: Neural Anisotropic Surface Meshing [38.8654207201197]
  This paper introduces a new learning-based method, NASM, for anisotropic surface meshing. Key idea is to embed an input mesh into a high-d Euclidean embedding space to preserve curvature-based anisotropic metric. Then, we propose a novel feature-sensitive remeshing on the generated high-d embedding to automatically capture sharp geometric features.
  arXiv  Detail & Related papers  (2024-10-30T15:20:10Z)
• Neural P$^3$M: A Long-Range Interaction Modeling Enhancer for Geometric GNNs [66.98487644676906]
  We introduce Neural P$3$M, a versatile enhancer of geometric GNNs to expand the scope of their capabilities. It exhibits flexibility across a wide range of molecular systems and demonstrates remarkable accuracy in predicting energies and forces. It also achieves an average improvement of 22% on the OE62 dataset while integrating with various architectures.
  arXiv  Detail & Related papers  (2024-09-26T08:16:59Z)
• N-BVH: Neural ray queries with bounding volume hierarchies [51.430495562430565]
  In 3D computer graphics, the bulk of a scene's memory usage is due to polygons and textures. We devise N-BVH, a neural compression architecture designed to answer arbitrary ray queries in 3D. Our method provides faithful approximations of visibility, depth, and appearance attributes.
  arXiv  Detail & Related papers  (2024-05-25T13:54:34Z)
• Learning Multimodal Volumetric Features for Large-Scale Neuron Tracing [72.45257414889478]
  We aim to reduce human workload by predicting connectivity between over-segmented neuron pieces. We first construct a dataset, named FlyTracing, that contains millions of pairwise connections of segments expanding the whole fly brain. We propose a novel connectivity-aware contrastive learning method to generate dense volumetric EM image embedding.
  arXiv  Detail & Related papers  (2024-01-05T19:45:12Z)
• Multi-view Hybrid Graph Convolutional Network for Volume-to-mesh Reconstruction in Cardiovascular MRI [43.47826598981827]
  We introduce HybridVNet, a novel architecture for direct image-to-mesh extraction. We show it can efficiently handle surface and volumetric meshes by encoding them as graph structures. Our model combines traditional convolutional networks with variational graph generative models, deep supervision and mesh-specific regularisation.
  arXiv  Detail & Related papers  (2023-11-22T21:51:29Z)
• A computational geometry approach for modeling neuronal fiber pathways [10.741721423684305]
  Tractography constructs complex neuronal fibers in three dimensions that exhibit the geometry of white matter pathways in the brain. We develop a computational geometry-based tractography representation that aims to simplify the connectivity of white matter fibers. Using diffusion MRI data from Alzheimer's patient study, we extract tractography features from our model for distinguishing the Alzheimer's subject from the normal control.
  arXiv  Detail & Related papers  (2021-08-02T21:16:29Z)
• GeoMol: Torsional Geometric Generation of Molecular 3D Conformer Ensembles [60.12186997181117]
  Prediction of a molecule's 3D conformer ensemble from the molecular graph holds a key role in areas of cheminformatics and drug discovery. Existing generative models have several drawbacks including lack of modeling important molecular geometry elements. We propose GeoMol, an end-to-end, non-autoregressive and SE(3)-invariant machine learning approach to generate 3D conformers.
  arXiv  Detail & Related papers  (2021-06-08T14:17:59Z)
• TSGCNet: Discriminative Geometric Feature Learning with Two-Stream GraphConvolutional Network for 3D Dental Model Segmentation [141.2690520327948]
  We propose a two-stream graph convolutional network (TSGCNet) to learn multi-view information from different geometric attributes. We evaluate our proposed TSGCNet on a real-patient dataset of dental models acquired by 3D intraoral scanners.
  arXiv  Detail & Related papers  (2020-12-26T08:02:56Z)
• Learning Deformable Tetrahedral Meshes for 3D Reconstruction [78.0514377738632]
  3D shape representations that accommodate learning-based 3D reconstruction are an open problem in machine learning and computer graphics. Previous work on neural 3D reconstruction demonstrated benefits, but also limitations, of point cloud, voxel, surface mesh, and implicit function representations. We introduce Deformable Tetrahedral Meshes (DefTet) as a particular parameterization that utilizes volumetric tetrahedral meshes for the reconstruction problem.
  arXiv  Detail & Related papers  (2020-11-03T02:57:01Z)
• Embed Me If You Can: A Geometric Perceptron [14.274582421372308]
  We introduce an extension of the multilayer hypersphere perceptron (MLHP) Our model is superior to the vanilla multilayer perceptron when classifying 3D Tetris shapes.
  arXiv  Detail & Related papers  (2020-06-11T15:25:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.