GeometricImageNet: Extending convolutional neural networks to vector and
tensor images
- URL: http://arxiv.org/abs/2305.12585v1
- Date: Sun, 21 May 2023 22:44:18 GMT
- Title: GeometricImageNet: Extending convolutional neural networks to vector and
tensor images
- Authors: Wilson Gregory, David W. Hogg, Ben Blum-Smith, Maria Teresa Arias,
Kaze W. K. Wong, Soledad Villar
- Abstract summary: GeometricImageNet is a generalization of convolution with outer products, tensor index contractions, and tensor index permutations.
In numerical experiments, we find that GeometricImageNet has good generalization for a small simulated physics system.
We expect this tool will be valuable for scientific and engineering machine learning, for example in cosmology or ocean dynamics.
- Score: 6.540771405203322
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Convolutional neural networks and their ilk have been very successful for
many learning tasks involving images. These methods assume that the input is a
scalar image representing the intensity in each pixel, possibly in multiple
channels for color images. In natural-science domains however, image-like data
sets might have vectors (velocity, say), tensors (polarization, say),
pseudovectors (magnetic field, say), or other geometric objects in each pixel.
Treating the components of these objects as independent channels in a CNN
neglects their structure entirely. Our formulation -- the GeometricImageNet --
combines a geometric generalization of convolution with outer products, tensor
index contractions, and tensor index permutations to construct geometric-image
functions of geometric images that use and benefit from the tensor structure.
The framework permits, with a very simple adjustment, restriction to function
spaces that are exactly equivariant to translations, discrete rotations, and
reflections. We use representation theory to quantify the dimension of the
space of equivariant polynomial functions on 2-dimensional vector images. We
give partial results on the expressivity of GeometricImageNet on small images.
In numerical experiments, we find that GeometricImageNet has good
generalization for a small simulated physics system, even when trained with a
small training set. We expect this tool will be valuable for scientific and
engineering machine learning, for example in cosmology or ocean dynamics.
Related papers
- Neural Geometry Processing via Spherical Neural Surfaces [29.30952578277242]
We show how to compute core geometric operators directly on a neural surface representation.
These operators, in turn, enable us to create geometry processing tools that act directly on the neural representations.
We demonstrate illustrative applications in (neural) spectral analysis, heat flow and mean curvature flow.
arXiv Detail & Related papers (2024-07-10T15:28:02Z) - Simplicial Representation Learning with Neural $k$-Forms [14.566552361705499]
This paper focuses on leveraging geometric information from simplicial complexes embedded in $mathbbRn$ using node coordinates.
We use differential k-forms in mathbbRn to create representations of simplices, offering interpretability and geometric consistency without message passing.
Our method is efficient, versatile, and applicable to various input complexes, including graphs, simplicial complexes, and cell complexes.
arXiv Detail & Related papers (2023-12-13T21:03:39Z) - A Hitchhiker's Guide to Geometric GNNs for 3D Atomic Systems [87.30652640973317]
Recent advances in computational modelling of atomic systems represent them as geometric graphs with atoms embedded as nodes in 3D Euclidean space.
Geometric Graph Neural Networks have emerged as the preferred machine learning architecture powering applications ranging from protein structure prediction to molecular simulations and material generation.
This paper provides a comprehensive and self-contained overview of the field of Geometric GNNs for 3D atomic systems.
arXiv Detail & Related papers (2023-12-12T18:44:19Z) - Geometric Clifford Algebra Networks [53.456211342585824]
We propose Geometric Clifford Algebra Networks (GCANs) for modeling dynamical systems.
GCANs are based on symmetry group transformations using geometric (Clifford) algebras.
arXiv Detail & Related papers (2023-02-13T18:48:33Z) - Prediction of Geometric Transformation on Cardiac MRI via Convolutional
Neural Network [13.01021780124613]
We propose to learn features in medical images by training ConvNets to recognize the geometric transformation applied to images.
We present a simple self-supervised task that can easily predict the geometric transformation.
arXiv Detail & Related papers (2022-11-12T11:29:14Z) - Towards General-Purpose Representation Learning of Polygonal Geometries [62.34832826705641]
We develop a general-purpose polygon encoding model, which can encode a polygonal geometry into an embedding space.
We conduct experiments on two tasks: 1) shape classification based on MNIST; 2) spatial relation prediction based on two new datasets - DBSR-46K and DBSR-cplx46K.
Our results show that NUFTspec and ResNet1D outperform multiple existing baselines with significant margins.
arXiv Detail & Related papers (2022-09-29T15:59:23Z) - A singular Riemannian geometry approach to Deep Neural Networks II.
Reconstruction of 1-D equivalence classes [78.120734120667]
We build the preimage of a point in the output manifold in the input space.
We focus for simplicity on the case of neural networks maps from n-dimensional real spaces to (n - 1)-dimensional real spaces.
arXiv Detail & Related papers (2021-12-17T11:47:45Z) - OSLO: On-the-Sphere Learning for Omnidirectional images and its
application to 360-degree image compression [59.58879331876508]
We study the learning of representation models for omnidirectional images and propose to use the properties of HEALPix uniform sampling of the sphere to redefine the mathematical tools used in deep learning models for omnidirectional images.
Our proposed on-the-sphere solution leads to a better compression gain that can save 13.7% of the bit rate compared to similar learned models applied to equirectangular images.
arXiv Detail & Related papers (2021-07-19T22:14:30Z) - The Intrinsic Dimension of Images and Its Impact on Learning [60.811039723427676]
It is widely believed that natural image data exhibits low-dimensional structure despite the high dimensionality of conventional pixel representations.
In this work, we apply dimension estimation tools to popular datasets and investigate the role of low-dimensional structure in deep learning.
arXiv Detail & Related papers (2021-04-18T16:29:23Z) - The Geometry of Deep Generative Image Models and its Applications [0.0]
Generative adversarial networks (GANs) have emerged as a powerful unsupervised method to model the statistical patterns of real-world data sets.
These networks are trained to map random inputs in their latent space to new samples representative of the learned data.
The structure of the latent space is hard to intuit due to its high dimensionality and the non-linearity of the generator.
arXiv Detail & Related papers (2021-01-15T07:57:33Z) - Learning Geometry-Dependent and Physics-Based Inverse Image
Reconstruction [9.565653662306806]
We present a new approach to learn inverse imaging that exploit the underlying geometry and physics.
We first introduce a non-Euclidean encoding-decoding network that allows us to describe the unknown and measurement variables.
We then learn the geometry-dependent physics in between the two domains by explicitly modeling it via a bipartite graph.
arXiv Detail & Related papers (2020-07-18T21:53:27Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.