Compressing multivariate functions with tree tensor networks

• URL: http://arxiv.org/abs/2410.03572v1
• Date: Fri, 4 Oct 2024 16:20:52 GMT
• Title: Compressing multivariate functions with tree tensor networks
• Authors: Joseph Tindall, Miles Stoudenmire, Ryan Levy,
• Abstract summary: One-dimensional tensor networks are increasingly being used as a numerical ansatz for continuum functions. We show how more structured tree tensor networks offer a significantly more efficient ansatz than the commonly used tensor train.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Tensor networks are a compressed format for multi-dimensional data. One-dimensional tensor networks -- often referred to as tensor trains (TT) or matrix product states (MPS) -- are increasingly being used as a numerical ansatz for continuum functions by "quantizing" the inputs into discrete binary digits. Here we demonstrate the power of more general tree tensor networks for this purpose. We provide direct constructions of a number of elementary functions as generic tree tensor networks and interpolative constructions for more complicated functions via a generalization of the tensor cross interpolation algorithm. For a range of multi-dimensional functions we show how more structured tree tensor networks offer a significantly more efficient ansatz than the commonly used tensor train. We demonstrate an application of our methods to solving multi-dimensional, non-linear Fredholm equations, providing a rigorous bound on the rank of the solution which, in turn, guarantees exponentially scaling accuracy with the size of the tree tensor network for certain problems.

Related papers

• One-step replica symmetry breaking in the language of tensor networks [0.913755431537592]
  We develop an exact mapping between the one-step replica symmetry breaking cavity method and tensor networks. The two schemes come with complementary mathematical and numerical toolboxes that could be leveraged to improve the respective states of the art.
  arXiv  Detail & Related papers  (2023-06-26T18:42:51Z)
• TensorKrowch: Smooth integration of tensor networks in machine learning [46.0920431279359]
  We introduceKrowch, an open source Python library built on top of PyTorch. Krowch allows users to construct any tensor network, train it, and integrate it as a layer in more intricate deep learning models.
  arXiv  Detail & Related papers  (2023-06-14T15:55:19Z)
• Low-Rank Tensor Function Representation for Multi-Dimensional Data Recovery [52.21846313876592]
  Low-rank tensor function representation (LRTFR) can continuously represent data beyond meshgrid with infinite resolution. We develop two fundamental concepts for tensor functions, i.e., the tensor function rank and low-rank tensor function factorization. Our method substantiates the superiority and versatility of our method as compared with state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-12-01T04:00:38Z)
• Tensor networks in machine learning [0.0]
  A tensor network is a decomposition used to express and approximate large arrays of data. A merger of tensor networks with machine learning is natural. Herein the network parameters are adjusted to learn or classify a data-set.
  arXiv  Detail & Related papers  (2022-07-06T18:00:00Z)
• Stack operation of tensor networks [10.86105335102537]
  We propose a mathematically rigorous definition for the tensor network stack approach. We illustrate the main ideas with the matrix product states based machine learning as an example.
  arXiv  Detail & Related papers  (2022-03-28T12:45:13Z)
• Quantum Annealing Algorithms for Boolean Tensor Networks [0.0]
  We introduce and analyze three general algorithms for Boolean tensor networks. We show can be expressed as a quadratic unconstrained binary optimization problem suitable for solving on a quantum annealer. We demonstrate that tensor with up to millions of elements can be decomposed efficiently using a DWave 2000Q quantum annealer.
  arXiv  Detail & Related papers  (2021-07-28T22:38:18Z)
• Connecting Weighted Automata, Tensor Networks and Recurrent Neural Networks through Spectral Learning [58.14930566993063]
  We present connections between three models used in different research fields: weighted finite automata(WFA) from formal languages and linguistics, recurrent neural networks used in machine learning, and tensor networks. We introduce the first provable learning algorithm for linear 2-RNN defined over sequences of continuous vectors input.
  arXiv  Detail & Related papers  (2020-10-19T15:28:00Z)
• T-Basis: a Compact Representation for Neural Networks [89.86997385827055]
  We introduce T-Basis, a concept for a compact representation of a set of tensors, each of an arbitrary shape, which is often seen in Neural Networks. We evaluate the proposed approach on the task of neural network compression and demonstrate that it reaches high compression rates at acceptable performance drops.
  arXiv  Detail & Related papers  (2020-07-13T19:03:22Z)
• Approximation with Tensor Networks. Part II: Approximation Rates for Smoothness Classes [0.0]
  We study the approximation by tensor networks (TNs) of functions from smoothness classes. The resulting tool can be interpreted as a feed-forward neural network. We show that arbitrary Besov functions can be approximated with optimal or near to optimal rate.
  arXiv  Detail & Related papers  (2020-06-30T21:57:42Z)
• Multipole Graph Neural Operator for Parametric Partial Differential Equations [57.90284928158383]
  One of the main challenges in using deep learning-based methods for simulating physical systems is formulating physics-based data. We propose a novel multi-level graph neural network framework that captures interaction at all ranges with only linear complexity. Experiments confirm our multi-graph network learns discretization-invariant solution operators to PDEs and can be evaluated in linear time.
  arXiv  Detail & Related papers  (2020-06-16T21:56:22Z)
• Neural Networks are Convex Regularizers: Exact Polynomial-time Convex Optimization Formulations for Two-layer Networks [70.15611146583068]
  We develop exact representations of training two-layer neural networks with rectified linear units (ReLUs) Our theory utilizes semi-infinite duality and minimum norm regularization.
  arXiv  Detail & Related papers  (2020-02-24T21:32:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.