LaB-GATr: geometric algebra transformers for large biomedical surface
and volume meshes
- URL: http://arxiv.org/abs/2403.07536v1
- Date: Tue, 12 Mar 2024 11:19:46 GMT
- Title: LaB-GATr: geometric algebra transformers for large biomedical surface
and volume meshes
- Authors: Julian Suk, Baris Imre, Jelmer M. Wolterink
- Abstract summary: We propose LaB-GATr, a deep neural network with geometric tokenisation for learning with high-fidelity meshes.
LaB-GATr achieves state-of-the-art results on three tasks in cardiovascular hemodynamics modelling and neurodevelopmental phenotype prediction.
Our results demonstrate that LaB-GATr is a powerful architecture for learning with high-fidelity meshes which has the potential to enable interesting downstream applications.
- Score: 1.6451639748812472
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Many anatomical structures can be described by surface or volume meshes.
Machine learning is a promising tool to extract information from these 3D
models. However, high-fidelity meshes often contain hundreds of thousands of
vertices, which creates unique challenges in building deep neural network
architectures. Furthermore, patient-specific meshes may not be canonically
aligned which limits the generalisation of machine learning algorithms. We
propose LaB-GATr, a transfomer neural network with geometric tokenisation that
can effectively learn with large-scale (bio-)medical surface and volume meshes
through sequence compression and interpolation. Our method extends the recently
proposed geometric algebra transformer (GATr) and thus respects all Euclidean
symmetries, i.e. rotation, translation and reflection, effectively mitigating
the problem of canonical alignment between patients. LaB-GATr achieves
state-of-the-art results on three tasks in cardiovascular hemodynamics
modelling and neurodevelopmental phenotype prediction, featuring meshes of up
to 200,000 vertices. Our results demonstrate that LaB-GATr is a powerful
architecture for learning with high-fidelity meshes which has the potential to
enable interesting downstream applications. Our implementation is publicly
available.
Related papers
- N-BVH: Neural ray queries with bounding volume hierarchies [51.430495562430565]
In 3D computer graphics, the bulk of a scene's memory usage is due to polygons and textures.
We devise N-BVH, a neural compression architecture designed to answer arbitrary ray queries in 3D.
Our method provides faithful approximations of visibility, depth, and appearance attributes.
arXiv Detail & Related papers (2024-05-25T13:54:34Z) - Learning Multimodal Volumetric Features for Large-Scale Neuron Tracing [72.45257414889478]
We aim to reduce human workload by predicting connectivity between over-segmented neuron pieces.
We first construct a dataset, named FlyTracing, that contains millions of pairwise connections of segments expanding the whole fly brain.
We propose a novel connectivity-aware contrastive learning method to generate dense volumetric EM image embedding.
arXiv Detail & Related papers (2024-01-05T19:45:12Z) - Multi-view Hybrid Graph Convolutional Network for Volume-to-mesh
Reconstruction in Cardiovascular MRI [44.53796049862948]
We introduce HybridVNet, a novel architecture for direct image-to-mesh extraction.
We show it can efficiently handle surface and volumetric meshes by encoding them as graph structures.
Our model combines traditional convolutional networks with variational graph generative models, deep supervision and mesh-specific regularisation.
arXiv Detail & Related papers (2023-11-22T21:51:29Z) - VesselVAE: Recursive Variational Autoencoders for 3D Blood Vessel
Synthesis [0.879967413208593]
We present a data-driven generative framework for synthesizing blood vessel 3D geometry.
VesselVAE fully exploits the hierarchical organization of the vessel and learns a low-dimensional manifold.
We generate 3D models of blood vessels that are both accurate and diverse, which is crucial for medical and surgical training, hemodynamic simulations, and many other purposes.
arXiv Detail & Related papers (2023-07-07T13:35:48Z) - A computational geometry approach for modeling neuronal fiber pathways [10.741721423684305]
Tractography constructs complex neuronal fibers in three dimensions that exhibit the geometry of white matter pathways in the brain.
We develop a computational geometry-based tractography representation that aims to simplify the connectivity of white matter fibers.
Using diffusion MRI data from Alzheimer's patient study, we extract tractography features from our model for distinguishing the Alzheimer's subject from the normal control.
arXiv Detail & Related papers (2021-08-02T21:16:29Z) - GeoMol: Torsional Geometric Generation of Molecular 3D Conformer
Ensembles [60.12186997181117]
Prediction of a molecule's 3D conformer ensemble from the molecular graph holds a key role in areas of cheminformatics and drug discovery.
Existing generative models have several drawbacks including lack of modeling important molecular geometry elements.
We propose GeoMol, an end-to-end, non-autoregressive and SE(3)-invariant machine learning approach to generate 3D conformers.
arXiv Detail & Related papers (2021-06-08T14:17:59Z) - TSGCNet: Discriminative Geometric Feature Learning with Two-Stream
GraphConvolutional Network for 3D Dental Model Segmentation [141.2690520327948]
We propose a two-stream graph convolutional network (TSGCNet) to learn multi-view information from different geometric attributes.
We evaluate our proposed TSGCNet on a real-patient dataset of dental models acquired by 3D intraoral scanners.
arXiv Detail & Related papers (2020-12-26T08:02:56Z) - Learning Deformable Tetrahedral Meshes for 3D Reconstruction [78.0514377738632]
3D shape representations that accommodate learning-based 3D reconstruction are an open problem in machine learning and computer graphics.
Previous work on neural 3D reconstruction demonstrated benefits, but also limitations, of point cloud, voxel, surface mesh, and implicit function representations.
We introduce Deformable Tetrahedral Meshes (DefTet) as a particular parameterization that utilizes volumetric tetrahedral meshes for the reconstruction problem.
arXiv Detail & Related papers (2020-11-03T02:57:01Z) - Hyperbolic Neural Networks++ [66.16106727715061]
We generalize the fundamental components of neural networks in a single hyperbolic geometry model, namely, the Poincar'e ball model.
Experiments show the superior parameter efficiency of our methods compared to conventional hyperbolic components, and stability and outperformance over their Euclidean counterparts.
arXiv Detail & Related papers (2020-06-15T08:23:20Z) - Embed Me If You Can: A Geometric Perceptron [14.274582421372308]
We introduce an extension of the multilayer hypersphere perceptron (MLHP)
Our model is superior to the vanilla multilayer perceptron when classifying 3D Tetris shapes.
arXiv Detail & Related papers (2020-06-11T15:25:50Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.