Entanglement and Tensor Networks for Supervised Image Classification
- URL: http://arxiv.org/abs/2007.06082v1
- Date: Sun, 12 Jul 2020 20:09:26 GMT
- Title: Entanglement and Tensor Networks for Supervised Image Classification
- Authors: John Martyn, Guifre Vidal, Chase Roberts, Stefan Leichenauer
- Abstract summary: We revisit the use of tensor networks for supervised image classification using the MNIST data set of digits of handwritten.
We propose a plausible candidate state $|Sigma_ellrangle$ and investigate its entanglement properties.
We conclude that $|Sigma_ellrangle$ is so robustly entangled that it cannot be approximated by the tensor network used in that work.
- Score: 0.0
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Tensor networks, originally designed to address computational problems in
quantum many-body physics, have recently been applied to machine learning
tasks. However, compared to quantum physics, where the reasons for the success
of tensor network approaches over the last 30 years is well understood, very
little is yet known about why these techniques work for machine learning. The
goal of this paper is to investigate entanglement properties of tensor network
models in a current machine learning application, in order to uncover general
principles that may guide future developments. We revisit the use of tensor
networks for supervised image classification using the MNIST data set of
handwritten digits, as pioneered by Stoudenmire and Schwab [Adv. in Neur.
Inform. Proc. Sys. 29, 4799 (2016)]. Firstly we hypothesize about which state
the tensor network might be learning during training. For that purpose, we
propose a plausible candidate state $|\Sigma_{\ell}\rangle$ (built as a
superposition of product states corresponding to images in the training set)
and investigate its entanglement properties. We conclude that
$|\Sigma_{\ell}\rangle$ is so robustly entangled that it cannot be approximated
by the tensor network used in that work, which must therefore be representing a
very different state. Secondly, we use tensor networks with a block product
structure, in which entanglement is restricted within small blocks of $n \times
n$ pixels/qubits. We find that these states are extremely expressive (e.g.
training accuracy of $99.97 \%$ already for $n=2$), suggesting that long-range
entanglement may not be essential for image classification. However, in our
current implementation, optimization leads to over-fitting, resulting in test
accuracies that are not competitive with other current approaches.
Related papers
- Neural Metamorphosis [72.88137795439407]
This paper introduces a new learning paradigm termed Neural Metamorphosis (NeuMeta), which aims to build self-morphable neural networks.
NeuMeta directly learns the continuous weight manifold of neural networks.
It sustains full-size performance even at a 75% compression rate.
arXiv Detail & Related papers (2024-10-10T14:49:58Z) - Machine learning with tree tensor networks, CP rank constraints, and
tensor dropout [6.385624548310884]
We show how tree tensor networks (TTN) with CP rank constraints and dropout tensor can be used in machine learning.
A low-rank TTN classifier with branching ratio $b=4$ reaches test set accuracy 90.3% with low computation costs.
arXiv Detail & Related papers (2023-05-30T22:22:24Z) - Near-Linear Time and Fixed-Parameter Tractable Algorithms for Tensor
Decompositions [51.19236668224547]
We study low rank approximation of tensors, focusing on the tensor train and Tucker decompositions.
For tensor train decomposition, we give a bicriteria $(1 + eps)$-approximation algorithm with a small bicriteria rank and $O(q cdot nnz(A))$ running time.
In addition, we extend our algorithm to tensor networks with arbitrary graphs.
arXiv Detail & Related papers (2022-07-15T11:55:09Z) - Tensor Network States with Low-Rank Tensors [6.385624548310884]
We introduce the idea of imposing low-rank constraints on the tensors that compose the tensor network.
With this modification, the time and complexities for the network optimization can be substantially reduced.
We find that choosing the tensor rank $r$ to be on the order of the bond $m$, is sufficient to obtain high-accuracy groundstate approximations.
arXiv Detail & Related papers (2022-05-30T17:58:16Z) - Stack operation of tensor networks [10.86105335102537]
We propose a mathematically rigorous definition for the tensor network stack approach.
We illustrate the main ideas with the matrix product states based machine learning as an example.
arXiv Detail & Related papers (2022-03-28T12:45:13Z) - Neural Capacitance: A New Perspective of Neural Network Selection via
Edge Dynamics [85.31710759801705]
Current practice requires expensive computational costs in model training for performance prediction.
We propose a novel framework for neural network selection by analyzing the governing dynamics over synaptic connections (edges) during training.
Our framework is built on the fact that back-propagation during neural network training is equivalent to the dynamical evolution of synaptic connections.
arXiv Detail & Related papers (2022-01-11T20:53:15Z) - Beyond Lazy Training for Over-parameterized Tensor Decomposition [69.4699995828506]
We show that gradient descent on over-parametrized objective could go beyond the lazy training regime and utilize certain low-rank structure in the data.
Our results show that gradient descent on over-parametrized objective could go beyond the lazy training regime and utilize certain low-rank structure in the data.
arXiv Detail & Related papers (2020-10-22T00:32:12Z) - Optimization at the boundary of the tensor network variety [2.1839191255085995]
tensor network states form a variational ansatz class widely used in the study of quantum many-body systems.
Recent work has shown that states on the boundary of this variety can yield more efficient representations for states of physical interest.
We show how to optimize over this class in order to find ground states of local Hamiltonians.
arXiv Detail & Related papers (2020-06-30T16:58:55Z) - Deep Polynomial Neural Networks [77.70761658507507]
$Pi$Nets are a new class of function approximators based on expansions.
$Pi$Nets produce state-the-art results in three challenging tasks, i.e. image generation, face verification and 3D mesh representation learning.
arXiv Detail & Related papers (2020-06-20T16:23:32Z) - $\Pi-$nets: Deep Polynomial Neural Networks [86.36557534288535]
$Pi$-Nets are neural networks in which the output is a high-order of the input.
We empirically demonstrate that $Pi$-Nets have better representation power than standard DCNNs.
Our framework elucidates why recent generative models, such as StyleGAN, improve upon their predecessors.
arXiv Detail & Related papers (2020-03-08T18:48:43Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.