Stack operation of tensor networks

• URL: http://arxiv.org/abs/2203.16338v1
• Date: Mon, 28 Mar 2022 12:45:13 GMT
• Title: Stack operation of tensor networks
• Authors: Tianning Zhang, L. K. Ang, Tianqi Chen, Bo Yang, Erping Li
• Abstract summary: We propose a mathematically rigorous definition for the tensor network stack approach. We illustrate the main ideas with the matrix product states based machine learning as an example.
• Score: 10.86105335102537
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The tensor network, as a facterization of tensors, aims at performing the operations that are common for normal tensors, such as addition, contraction and stacking. However, due to its non-unique network structure, only the tensor network contraction is so far well defined. In this paper, we propose a mathematically rigorous definition for the tensor network stack approach, that compress a large amount of tensor networks into a single one without changing their structures and configurations. We illustrate the main ideas with the matrix product states based machine learning as an example. Our results are compared with the for loop and the efficient coding method on both CPU and GPU.

Related papers

• Compressing multivariate functions with tree tensor networks [0.0]
  One-dimensional tensor networks are increasingly being used as a numerical ansatz for continuum functions. We show how more structured tree tensor networks offer a significantly more efficient ansatz than the commonly used tensor train.
  arXiv  Detail & Related papers  (2024-10-04T16:20:52Z)
• An introduction to graphical tensor notation for mechanistic interpretability [0.0]
  It's often easy to get confused about which operations are happening between tensors. The first half of this document introduces the notation and applies it to some decompositions. The second half applies it to some existing some foundational approaches for mechanistically understanding language models.
  arXiv  Detail & Related papers  (2024-02-02T02:56:01Z)
• Symbolically integrating tensor networks over various random tensors by the second version of Python RTNI [0.5439020425818999]
  We are upgrading the Python-version of RTNI, which symbolically integrates tensor networks over the Haar-distributed unitary matrices. Now, PyRTNI2 can treat the Haar-distributed matrices and the real and complex normal Gaussian tensors as well. In this paper, we explain maths behind the program and show what kind of tensor network calculations can be made with it.
  arXiv  Detail & Related papers  (2023-09-03T13:14:46Z)
• TensorKrowch: Smooth integration of tensor networks in machine learning [46.0920431279359]
  We introduceKrowch, an open source Python library built on top of PyTorch. Krowch allows users to construct any tensor network, train it, and integrate it as a layer in more intricate deep learning models.
  arXiv  Detail & Related papers  (2023-06-14T15:55:19Z)
• Logits are predictive of network type [47.64219291655723]
  It is possible to predict which deep network has generated a given logit vector with accuracy well above chance. We utilize a number of networks on a dataset, with random weights or pretrained weights, as well as fine-tuned networks.
  arXiv  Detail & Related papers  (2022-11-04T05:53:27Z)
• Near-Linear Time and Fixed-Parameter Tractable Algorithms for Tensor Decompositions [51.19236668224547]
  We study low rank approximation of tensors, focusing on the tensor train and Tucker decompositions. For tensor train decomposition, we give a bicriteria $(1 + eps)$-approximation algorithm with a small bicriteria rank and $O(q cdot nnz(A))$ running time. In addition, we extend our algorithm to tensor networks with arbitrary graphs.
  arXiv  Detail & Related papers  (2022-07-15T11:55:09Z)
• Tensor networks in machine learning [0.0]
  A tensor network is a decomposition used to express and approximate large arrays of data. A merger of tensor networks with machine learning is natural. Herein the network parameters are adjusted to learn or classify a data-set.
  arXiv  Detail & Related papers  (2022-07-06T18:00:00Z)
• DeepReduce: A Sparse-tensor Communication Framework for Distributed Deep Learning [79.89085533866071]
  This paper introduces DeepReduce, a versatile framework for the compressed communication of sparse tensors. DeepReduce decomposes tensors in two sets, values and indices, and allows both independent and combined compression of these sets. Our experiments with large real models demonstrate that DeepReduce transmits fewer data and imposes lower computational overhead than existing methods.
  arXiv  Detail & Related papers  (2021-02-05T11:31:24Z)
• Permute, Quantize, and Fine-tune: Efficient Compression of Neural Networks [70.0243910593064]
  Key to success of vector quantization is deciding which parameter groups should be compressed together. In this paper we make the observation that the weights of two adjacent layers can be permuted while expressing the same function. We then establish a connection to rate-distortion theory and search for permutations that result in networks that are easier to compress.
  arXiv  Detail & Related papers  (2020-10-29T15:47:26Z)
• T-Basis: a Compact Representation for Neural Networks [89.86997385827055]
  We introduce T-Basis, a concept for a compact representation of a set of tensors, each of an arbitrary shape, which is often seen in Neural Networks. We evaluate the proposed approach on the task of neural network compression and demonstrate that it reaches high compression rates at acceptable performance drops.
  arXiv  Detail & Related papers  (2020-07-13T19:03:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.