Statistical Mechanics of Neural Processing of Object Manifolds
- URL: http://arxiv.org/abs/2106.00790v1
- Date: Tue, 1 Jun 2021 20:49:14 GMT
- Title: Statistical Mechanics of Neural Processing of Object Manifolds
- Authors: SueYeon Chung
- Abstract summary: This thesis lays the groundwork for a computational theory of neuronal processing of objects.
We identify that the capacity of a manifold is determined that effective radius, R_M, and effective dimension, D_M.
- Score: 3.4809730725241605
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Invariant object recognition is one of the most fundamental cognitive tasks
performed by the brain. In the neural state space, different objects with
stimulus variabilities are represented as different manifolds. In this
geometrical perspective, object recognition becomes the problem of linearly
separating different object manifolds. In feedforward visual hierarchy, it has
been suggested that the object manifold representations are reformatted across
the layers, to become more linearly separable. Thus, a complete theory of
perception requires characterizing the ability of linear readout networks to
classify object manifolds from variable neural responses.
A theory of the perceptron of isolated points was pioneered by E. Gardner who
formulated it as a statistical mechanics problem and analyzed it using replica
theory. In this thesis, we generalize Gardner's analysis and establish a theory
of linear classification of manifolds synthesizing statistical and geometric
properties of high dimensional signals. [..] Next, we generalize our theory
further to linear classification of general perceptual manifolds, such as point
clouds. We identify that the capacity of a manifold is determined that
effective radius, R_M, and effective dimension, D_M. Finally, we show
extensions relevant for applications to real data, incorporating correlated
manifolds, heterogenous manifold geometries, sparse labels and nonlinear
classifications. Then, we demonstrate how object-based manifolds transform in
standard deep networks.
This thesis lays the groundwork for a computational theory of neuronal
processing of objects, providing quantitative measures for linear separability
of object manifolds. We hope this theory will provide new insights into the
computational principles underlying processing of sensory representations in
biological and artificial neural networks.
Related papers
- A Theoretical Study of Neural Network Expressive Power via Manifold Topology [9.054396245059555]
A prevalent assumption regarding real-world data is that it lies on or close to a low-dimensional manifold.
In this study, we investigate network expressive power in terms of the latent data manifold.
We present a size upper bound of ReLU neural networks.
arXiv Detail & Related papers (2024-10-21T22:10:24Z) - On the Geometry and Optimization of Polynomial Convolutional Networks [2.9816332334719773]
We study convolutional neural networks with monomial activation functions.
We compute the dimension and the degree of the neuromanifold, which measure the expressivity of the model.
For a generic large dataset, we derive an explicit formula that quantifies the number of critical points arising in the optimization of a regression loss.
arXiv Detail & Related papers (2024-10-01T14:13:05Z) - Geometric Understanding of Discriminability and Transferability for Visual Domain Adaptation [27.326817457760725]
Invariant representation learning for unsupervised domain adaptation (UDA) has made significant advances in computer vision and pattern recognition communities.
Recently, empirical connections between transferability and discriminability have received increasing attention.
In this work, we systematically analyze the essentials of transferability and discriminability from the geometric perspective.
arXiv Detail & Related papers (2024-06-24T13:31:08Z) - Learning Discrete Concepts in Latent Hierarchical Models [73.01229236386148]
Learning concepts from natural high-dimensional data holds potential in building human-aligned and interpretable machine learning models.
We formalize concepts as discrete latent causal variables that are related via a hierarchical causal model.
We substantiate our theoretical claims with synthetic data experiments.
arXiv Detail & Related papers (2024-06-01T18:01:03Z) - A Hitchhiker's Guide to Geometric GNNs for 3D Atomic Systems [87.30652640973317]
Recent advances in computational modelling of atomic systems represent them as geometric graphs with atoms embedded as nodes in 3D Euclidean space.
Geometric Graph Neural Networks have emerged as the preferred machine learning architecture powering applications ranging from protein structure prediction to molecular simulations and material generation.
This paper provides a comprehensive and self-contained overview of the field of Geometric GNNs for 3D atomic systems.
arXiv Detail & Related papers (2023-12-12T18:44:19Z) - Unsupervised Learning of Invariance Transformations [105.54048699217668]
We develop an algorithmic framework for finding approximate graph automorphisms.
We discuss how this framework can be used to find approximate automorphisms in weighted graphs in general.
arXiv Detail & Related papers (2023-07-24T17:03:28Z) - Towards a mathematical understanding of learning from few examples with
nonlinear feature maps [68.8204255655161]
We consider the problem of data classification where the training set consists of just a few data points.
We reveal key relationships between the geometry of an AI model's feature space, the structure of the underlying data distributions, and the model's generalisation capabilities.
arXiv Detail & Related papers (2022-11-07T14:52:58Z) - The Manifold Scattering Transform for High-Dimensional Point Cloud Data [16.500568323161563]
We present practical schemes for implementing the manifold scattering transform to datasets arising in naturalistic systems.
We show that our methods are effective for signal classification and manifold classification tasks.
arXiv Detail & Related papers (2022-06-21T02:15:00Z) - Survey: Geometric Foundations of Data Reduction [2.238700807267101]
The purpose of this survey is to briefly introduce nonlinear dimensionality reduction (NLDR) in data reduction.
In 2001, the concept of Manifold Learning first appears as an NLDR method called Laplacian Eigenmaps.
We derive each spectral manifold learning with the matrix and operator representation, and we then discuss the convergence behavior of each method in a geometric uniform language.
arXiv Detail & Related papers (2020-08-16T07:59:22Z) - Disentangling by Subspace Diffusion [72.1895236605335]
We show that fully unsupervised factorization of a data manifold is possible if the true metric of the manifold is known.
Our work reduces the question of whether unsupervised metric learning is possible, providing a unifying insight into the geometric nature of representation learning.
arXiv Detail & Related papers (2020-06-23T13:33:19Z) - Learning Bijective Feature Maps for Linear ICA [73.85904548374575]
We show that existing probabilistic deep generative models (DGMs) which are tailor-made for image data, underperform on non-linear ICA tasks.
To address this, we propose a DGM which combines bijective feature maps with a linear ICA model to learn interpretable latent structures for high-dimensional data.
We create models that converge quickly, are easy to train, and achieve better unsupervised latent factor discovery than flow-based models, linear ICA, and Variational Autoencoders on images.
arXiv Detail & Related papers (2020-02-18T17:58:07Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.