Residual Matrix Product State for Machine Learning

• URL: http://arxiv.org/abs/2012.11841v1
• Date: Tue, 22 Dec 2020 05:44:20 GMT
• Title: Residual Matrix Product State for Machine Learning
• Authors: Ye-Ming Meng, Jing Zhang, Peng Zhang, Chao Gao and Shi-Ju Ran
• Abstract summary: We propose the residual matrix product state (ResMPS) by combining the ideas of matrix product state (MPS) and residual neural network (NN) ResMPS can be treated as a network where its layers map the "hidden" features to the outputs. It outperforms state-of-the-art TN models on efficiency, stability and expression power.
• Score: 20.158215120846652
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Tensor network (TN), which originates from quantum physics, shows broad prospects in classical and quantum machine learning (ML). However, there still exists a considerable gap of accuracy between TN and the sophisticated neural network (NN) models for classical ML. It is still elusive how far TN ML can be improved by, e.g., borrowing the techniques from NN. In this work, we propose the residual matrix product state (ResMPS) by combining the ideas of matrix product state (MPS) and residual NN. ResMPS can be treated as a network where its layers map the "hidden" features to the outputs (e.g., classifications), and the variational parameters of the layers are the functions of the features of samples (e.g., pixels of images). This is essentially different from NN, where the layers map feed-forwardly the features to the output. ResMPS can naturally incorporate with the non-linear activations and dropout layers, and outperforms the state-of-the-art TN models on the efficiency, stability, and expression power. Besides, ResMPS is interpretable from the perspective of polynomial expansion, where the factorization and exponential machines naturally emerge. Our work contributes to connecting and hybridizing neural and tensor networks, which is crucial to understand the working mechanisms further and improve both models' performances.

Related papers

• Deep Tree Tensor Networks for Image Recognition [1.8434042562191815]
  This paper introduces a novel architecture named ittextbfDeep textbfTree textbfTensor textbfNetwork (DTTN) DTTN captures $2L$-order multiplicative interactions across features through multilinear operations. We theoretically reveal the equivalency among quantum-inspired TN models and interacting networks under certain conditions.
  arXiv  Detail & Related papers  (2025-02-14T05:41:33Z)
• Scalable Mechanistic Neural Networks [52.28945097811129]
  We propose an enhanced neural network framework designed for scientific machine learning applications involving long temporal sequences. By reformulating the original Mechanistic Neural Network (MNN) we reduce the computational time and space complexities from cubic and quadratic with respect to the sequence length, respectively, to linear. Extensive experiments demonstrate that S-MNN matches the original MNN in precision while substantially reducing computational resources.
  arXiv  Detail & Related papers  (2024-10-08T14:27:28Z)
• A Matrix Product State Model for Simultaneous Classification and Generation [0.8192907805418583]
  Quantum machine learning (QML) is a rapidly expanding field that merges the principles of quantum computing with the techniques of machine learning. Here, we present a novel matrix product state (MPS) model, where the MPS functions as both a classifier and a generator. Our contributions offer insights into the mechanics of tensor network methods for generation tasks.
  arXiv  Detail & Related papers  (2024-06-25T10:23:36Z)
• Equivariant Matrix Function Neural Networks [1.8717045355288808]
  We introduce Matrix Function Neural Networks (MFNs), a novel architecture that parameterizes non-local interactions through analytic matrix equivariant functions. MFNs is able to capture intricate non-local interactions in quantum systems, paving the way to new state-of-the-art force fields.
  arXiv  Detail & Related papers  (2023-10-16T14:17:00Z)
• Neural Functional Transformers [99.98750156515437]
  This paper uses the attention mechanism to define a novel set of permutation equivariant weight-space layers called neural functional Transformers (NFTs) NFTs respect weight-space permutation symmetries while incorporating the advantages of attention, which have exhibited remarkable success across multiple domains. We also leverage NFTs to develop Inr2Array, a novel method for computing permutation invariant representations from the weights of implicit neural representations (INRs)
  arXiv  Detail & Related papers  (2023-05-22T23:38:27Z)
• Towards Neural Variational Monte Carlo That Scales Linearly with System Size [67.09349921751341]
  Quantum many-body problems are central to demystifying some exotic quantum phenomena, e.g., high-temperature superconductors. The combination of neural networks (NN) for representing quantum states, and the Variational Monte Carlo (VMC) algorithm, has been shown to be a promising method for solving such problems. We propose a NN architecture called Vector-Quantized Neural Quantum States (VQ-NQS) that utilizes vector-quantization techniques to leverage redundancies in the local-energy calculations of the VMC algorithm.
  arXiv  Detail & Related papers  (2022-12-21T19:00:04Z)
• BiTAT: Neural Network Binarization with Task-dependent Aggregated Transformation [116.26521375592759]
  Quantization aims to transform high-precision weights and activations of a given neural network into low-precision weights/activations for reduced memory usage and computation. Extreme quantization (1-bit weight/1-bit activations) of compactly-designed backbone architectures results in severe performance degeneration. This paper proposes a novel Quantization-Aware Training (QAT) method that can effectively alleviate performance degeneration.
  arXiv  Detail & Related papers  (2022-07-04T13:25:49Z)
• Universal approximation property of invertible neural networks [76.95927093274392]
  Invertible neural networks (INNs) are neural network architectures with invertibility by design. Thanks to their invertibility and the tractability of Jacobian, INNs have various machine learning applications such as probabilistic modeling, generative modeling, and representation learning.
  arXiv  Detail & Related papers  (2022-04-15T10:45:26Z)
• Variational learning of quantum ground states on spiking neuromorphic hardware [0.0]
  High-dimensional sampling spaces and transient autocorrelations confront neural networks with a challenging computational bottleneck. Compared to conventional neural networks, physical-model devices offer a fast, efficient and inherently parallel substrate. We demonstrate the ability of a neuromorphic chip to represent the ground states of quantum spin models by variational energy minimization.
  arXiv  Detail & Related papers  (2021-09-30T14:39:45Z)
• Representation Theorem for Matrix Product States [1.7894377200944511]
  We investigate the universal representation capacity of the Matrix Product States (MPS) from the perspective of functions and continuous functions. We show that MPS can accurately realize arbitrary functions by providing a construction method of the corresponding MPS structure for an arbitrarily given gate. We study the relation between MPS and neural networks and show that the MPS with a scale-invariant sigmoidal function is equivalent to a one-hidden-layer neural network.
  arXiv  Detail & Related papers  (2021-03-15T11:06:54Z)
• Flexible Transmitter Network [84.90891046882213]
  Current neural networks are mostly built upon the MP model, which usually formulates the neuron as executing an activation function on the real-valued weighted aggregation of signals received from other neurons. We propose the Flexible Transmitter (FT) model, a novel bio-plausible neuron model with flexible synaptic plasticity. We present the Flexible Transmitter Network (FTNet), which is built on the most common fully-connected feed-forward architecture.
  arXiv  Detail & Related papers  (2020-04-08T06:55:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.