Statistical Mechanics of Neural Processing of Object Manifolds
- URL: http://arxiv.org/abs/2106.00790v1
- Date: Tue, 1 Jun 2021 20:49:14 GMT
- Title: Statistical Mechanics of Neural Processing of Object Manifolds
- Authors: SueYeon Chung
- Abstract summary: This thesis lays the groundwork for a computational theory of neuronal processing of objects.
We identify that the capacity of a manifold is determined that effective radius, R_M, and effective dimension, D_M.
- Score: 3.4809730725241605
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Invariant object recognition is one of the most fundamental cognitive tasks
performed by the brain. In the neural state space, different objects with
stimulus variabilities are represented as different manifolds. In this
geometrical perspective, object recognition becomes the problem of linearly
separating different object manifolds. In feedforward visual hierarchy, it has
been suggested that the object manifold representations are reformatted across
the layers, to become more linearly separable. Thus, a complete theory of
perception requires characterizing the ability of linear readout networks to
classify object manifolds from variable neural responses.
A theory of the perceptron of isolated points was pioneered by E. Gardner who
formulated it as a statistical mechanics problem and analyzed it using replica
theory. In this thesis, we generalize Gardner's analysis and establish a theory
of linear classification of manifolds synthesizing statistical and geometric
properties of high dimensional signals. [..] Next, we generalize our theory
further to linear classification of general perceptual manifolds, such as point
clouds. We identify that the capacity of a manifold is determined that
effective radius, R_M, and effective dimension, D_M. Finally, we show
extensions relevant for applications to real data, incorporating correlated
manifolds, heterogenous manifold geometries, sparse labels and nonlinear
classifications. Then, we demonstrate how object-based manifolds transform in
standard deep networks.
This thesis lays the groundwork for a computational theory of neuronal
processing of objects, providing quantitative measures for linear separability
of object manifolds. We hope this theory will provide new insights into the
computational principles underlying processing of sensory representations in
biological and artificial neural networks.
Related papers
- Revealing Bias Formation in Deep Neural Networks Through the Geometric Mechanisms of Human Visual Decoupling [9.609083308026786]
Deep neural networks (DNNs) often exhibit biases toward certain categories during object recognition.
We propose a geometric analysis framework linking the geometric complexity of class-specific perceptual Manifolds to model bias.
We present the Perceptual-Manifold-Geometry library, designed for calculating the geometric properties of perceptual Manifolds.
arXiv Detail & Related papers (2025-02-17T13:54:02Z) - A Theoretical Study of Neural Network Expressive Power via Manifold Topology [9.054396245059555]
A prevalent assumption regarding real-world data is that it lies on or close to a low-dimensional manifold.
In this study, we investigate network expressive power in terms of the latent data manifold.
We present a size upper bound of ReLU neural networks.
arXiv Detail & Related papers (2024-10-21T22:10:24Z) - Range, not Independence, Drives Modularity in Biological Inspired Representation [52.48094670415497]
We develop a theory of when biologically inspired networks modularise their representation of source variables (sources)
We derive necessary and sufficient conditions on a sample of sources that determine whether the neurons in an optimal linear autoencoder modularise.
Our theory applies to any dataset, extending far beyond the case of statistical independence studied in previous work.
arXiv Detail & Related papers (2024-10-08T17:41:37Z) - On the Geometry and Optimization of Polynomial Convolutional Networks [2.9816332334719773]
We study convolutional neural networks with monomial activation functions.
We compute the dimension and the degree of the neuromanifold, which measure the expressivity of the model.
For a generic large dataset, we derive an explicit formula that quantifies the number of critical points arising in the optimization of a regression loss.
arXiv Detail & Related papers (2024-10-01T14:13:05Z) - Learning Discrete Concepts in Latent Hierarchical Models [73.01229236386148]
Learning concepts from natural high-dimensional data holds potential in building human-aligned and interpretable machine learning models.
We formalize concepts as discrete latent causal variables that are related via a hierarchical causal model.
We substantiate our theoretical claims with synthetic data experiments.
arXiv Detail & Related papers (2024-06-01T18:01:03Z) - Unsupervised Learning of Invariance Transformations [105.54048699217668]
We develop an algorithmic framework for finding approximate graph automorphisms.
We discuss how this framework can be used to find approximate automorphisms in weighted graphs in general.
arXiv Detail & Related papers (2023-07-24T17:03:28Z) - Towards a mathematical understanding of learning from few examples with
nonlinear feature maps [68.8204255655161]
We consider the problem of data classification where the training set consists of just a few data points.
We reveal key relationships between the geometry of an AI model's feature space, the structure of the underlying data distributions, and the model's generalisation capabilities.
arXiv Detail & Related papers (2022-11-07T14:52:58Z) - The Manifold Scattering Transform for High-Dimensional Point Cloud Data [16.500568323161563]
We present practical schemes for implementing the manifold scattering transform to datasets arising in naturalistic systems.
We show that our methods are effective for signal classification and manifold classification tasks.
arXiv Detail & Related papers (2022-06-21T02:15:00Z) - Survey: Geometric Foundations of Data Reduction [2.238700807267101]
The purpose of this survey is to briefly introduce nonlinear dimensionality reduction (NLDR) in data reduction.
In 2001, the concept of Manifold Learning first appears as an NLDR method called Laplacian Eigenmaps.
We derive each spectral manifold learning with the matrix and operator representation, and we then discuss the convergence behavior of each method in a geometric uniform language.
arXiv Detail & Related papers (2020-08-16T07:59:22Z) - Disentangling by Subspace Diffusion [72.1895236605335]
We show that fully unsupervised factorization of a data manifold is possible if the true metric of the manifold is known.
Our work reduces the question of whether unsupervised metric learning is possible, providing a unifying insight into the geometric nature of representation learning.
arXiv Detail & Related papers (2020-06-23T13:33:19Z) - Learning Bijective Feature Maps for Linear ICA [73.85904548374575]
We show that existing probabilistic deep generative models (DGMs) which are tailor-made for image data, underperform on non-linear ICA tasks.
To address this, we propose a DGM which combines bijective feature maps with a linear ICA model to learn interpretable latent structures for high-dimensional data.
We create models that converge quickly, are easy to train, and achieve better unsupervised latent factor discovery than flow-based models, linear ICA, and Variational Autoencoders on images.
arXiv Detail & Related papers (2020-02-18T17:58:07Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.