Grounding Continuous Representations in Geometry: Equivariant Neural Fields

• URL: http://arxiv.org/abs/2406.05753v4
• Date: Fri, 04 Oct 2024 15:00:24 GMT
• Title: Grounding Continuous Representations in Geometry: Equivariant Neural Fields
• Authors: David R Wessels, David M Knigge, Samuele Papa, Riccardo Valperga, Sharvaree Vadgama, Efstratios Gavves, Erik J Bekkers,
• Abstract summary: We propose a novel CNF architecture which uses a geometry-informed cross-attention to condition the NeF on a geometric variable. We show that this approach induces a steerability property by which both field and latent are grounded in geometry. We validate these main properties in a range of tasks including classification, segmentation, forecasting and reconstruction.
• Score: 26.567143650213225
• License:
• Abstract: Conditional Neural Fields (CNFs) are increasingly being leveraged as continuous signal representations, by associating each data-sample with a latent variable that conditions a shared backbone Neural Field (NeF) to reconstruct the sample. However, existing CNF architectures face limitations when using this latent downstream in tasks requiring fine grained geometric reasoning, such as classification and segmentation. We posit that this results from lack of explicit modelling of geometric information (e.g. locality in the signal or the orientation of a feature) in the latent space of CNFs. As such, we propose Equivariant Neural Fields (ENFs), a novel CNF architecture which uses a geometry-informed cross-attention to condition the NeF on a geometric variable, a latent point cloud of features, that enables an equivariant decoding from latent to field. We show that this approach induces a steerability property by which both field and latent are grounded in geometry and amenable to transformation laws: if the field transforms, the latent representation transforms accordingly - and vice versa. Crucially, this equivariance relation ensures that the latent is capable of (1) representing geometric patterns faitfhully, allowing for geometric reasoning in latent space, (2) weight-sharing over similar local patterns, allowing for efficient learning of datasets of fields. We validate these main properties in a range of tasks including classification, segmentation, forecasting and reconstruction, showing clear improvement over baselines with a geometry-free latent space.

Related papers

• Geometric Neural Process Fields [58.77241763774756]
  Geometric Neural Process Fields (G-NPF) is a probabilistic framework for neural radiance fields that explicitly captures uncertainty. Building on these bases, we design a hierarchical latent variable model, allowing G-NPF to integrate structural information across multiple spatial levels. Experiments on novel-view synthesis for 3D scenes, as well as 2D image and 1D signal regression, demonstrate the effectiveness of our method.
  arXiv  Detail & Related papers  (2025-02-04T14:17:18Z)
• SpaceMesh: A Continuous Representation for Learning Manifold Surface Meshes [61.110517195874074]
  We present a scheme to directly generate manifold, polygonal meshes of complex connectivity as the output of a neural network. Our key innovation is to define a continuous latent connectivity space at each mesh, which implies the discrete mesh. In applications, this approach not only yields high-quality outputs from generative models, but also enables directly learning challenging geometry processing tasks such as mesh repair.
  arXiv  Detail & Related papers  (2024-09-30T17:59:03Z)
• Geometric Neural Diffusion Processes [55.891428654434634]
  We extend the framework of diffusion models to incorporate a series of geometric priors in infinite-dimension modelling. We show that with these conditions, the generative functional model admits the same symmetry.
  arXiv  Detail & Related papers  (2023-07-11T16:51:38Z)
• Relative representations enable zero-shot latent space communication [19.144630518400604]
  Neural networks embed the geometric structure of a data manifold lying in a high-dimensional space into latent representations. We show how neural architectures can leverage these relative representations to guarantee, in practice, latent isometry invariance.
  arXiv  Detail & Related papers  (2022-09-30T12:37:03Z)
• Towards General-Purpose Representation Learning of Polygonal Geometries [62.34832826705641]
  We develop a general-purpose polygon encoding model, which can encode a polygonal geometry into an embedding space. We conduct experiments on two tasks: 1) shape classification based on MNIST; 2) spatial relation prediction based on two new datasets - DBSR-46K and DBSR-cplx46K. Our results show that NUFTspec and ResNet1D outperform multiple existing baselines with significant margins.
  arXiv  Detail & Related papers  (2022-09-29T15:59:23Z)
• Representing Deep Neural Networks Latent Space Geometries with Graphs [38.63434325489782]
  Deep Learning (DL) has attracted a lot of attention for its ability to reach state-of-the-art performance in many machine learning tasks. In this work, we show that it is possible to introduce constraints on these latent geometries to address various problems.
  arXiv  Detail & Related papers  (2020-11-14T17:21:29Z)
• A Point-Cloud Deep Learning Framework for Prediction of Fluid Flow Fields on Irregular Geometries [62.28265459308354]
  Network learns end-to-end mapping between spatial positions and CFD quantities. Incompress laminar steady flow past a cylinder with various shapes for its cross section is considered. Network predicts the flow fields hundreds of times faster than our conventional CFD.
  arXiv  Detail & Related papers  (2020-10-15T12:15:02Z)
• Unsupervised Geometric Disentanglement for Surfaces via CFAN-VAE [7.5714374873825765]
  We propose CFAN-VAE, a novel architecture that disentangles identity and pose using the CFAN feature. Requiring no label information on the identity or pose during training, CFAN-VAE achieves geometric disentanglement in an unsupervisedway. We also successfully detect a level of geometric disentanglement in mesh convolutional autoencoders that encode xyz-coordinates directly by registering its latent space to that of CFAN-VAE.
  arXiv  Detail & Related papers  (2020-05-23T23:28:10Z)
• Neural Subdivision [58.97214948753937]
  This paper introduces Neural Subdivision, a novel framework for data-driven coarseto-fine geometry modeling. We optimize for the same set of network weights across all local mesh patches, thus providing an architecture that is not constrained to a specific input mesh, fixed genus, or category. We demonstrate that even when trained on a single high-resolution mesh our method generates reasonable subdivisions for novel shapes.
  arXiv  Detail & Related papers  (2020-05-04T20:03:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.