Comb Tensor Networks vs. Matrix Product States: Enhanced Efficiency in High-Dimensional Spaces

• URL: http://arxiv.org/abs/2412.06857v1
• Date: Sun, 08 Dec 2024 20:28:49 GMT
• Title: Comb Tensor Networks vs. Matrix Product States: Enhanced Efficiency in High-Dimensional Spaces
• Authors: Danylo Kolesnyk, Yelyzaveta Vodovozova,
• Abstract summary: We show that a comb-shaped tensor network architecture can yield more efficient contractions than a standard MPS. This finding suggests that for continuous and high-dimensional data distributions, transitioning from MPS to a comb tensor network representation can substantially reduce computational overhead while maintaining accuracy.
• Score: 0.0
• License:
• Abstract: Modern approaches to generative modeling of continuous data using tensor networks incorporate compression layers to capture the most meaningful features of high-dimensional inputs. These methods, however, rely on traditional Matrix Product States (MPS) architectures. Here, we demonstrate that beyond a certain threshold in data and bond dimensions, a comb-shaped tensor network architecture can yield more efficient contractions than a standard MPS. This finding suggests that for continuous and high-dimensional data distributions, transitioning from MPS to a comb tensor network representation can substantially reduce computational overhead while maintaining accuracy.

Related papers

• Generalized Factor Neural Network Model for High-dimensional Regression [50.554377879576066]
  We tackle the challenges of modeling high-dimensional data sets with latent low-dimensional structures hidden within complex, non-linear, and noisy relationships. Our approach enables a seamless integration of concepts from non-parametric regression, factor models, and neural networks for high-dimensional regression.
  arXiv  Detail & Related papers  (2025-02-16T23:13:55Z)
• Parallel Sequence Modeling via Generalized Spatial Propagation Network [80.66202109995726]
  Generalized Spatial Propagation Network (GSPN) is a new attention mechanism for optimized vision tasks that inherently captures 2D spatial structures. GSPN overcomes limitations by directly operating on spatially coherent image data and forming dense pairwise connections through a line-scan approach. GSPN achieves superior spatial fidelity and state-of-the-art performance in vision tasks, including ImageNet classification, class-guided image generation, and text-to-image generation.
  arXiv  Detail & Related papers  (2025-01-21T18:56:19Z)
• SpaceMesh: A Continuous Representation for Learning Manifold Surface Meshes [61.110517195874074]
  We present a scheme to directly generate manifold, polygonal meshes of complex connectivity as the output of a neural network. Our key innovation is to define a continuous latent connectivity space at each mesh, which implies the discrete mesh. In applications, this approach not only yields high-quality outputs from generative models, but also enables directly learning challenging geometry processing tasks such as mesh repair.
  arXiv  Detail & Related papers  (2024-09-30T17:59:03Z)
• Adaptive Error-Bounded Hierarchical Matrices for Efficient Neural Network Compression [0.0]
  This paper introduces a dynamic, error-bounded hierarchical matrix (H-matrix) compression method tailored for Physics-Informed Neural Networks (PINNs) The proposed approach reduces the computational complexity and memory demands of large-scale physics-based models while preserving the essential properties of the Neural Tangent Kernel (NTK) Empirical results demonstrate that this technique outperforms traditional compression methods, such as Singular Value Decomposition (SVD), pruning, and quantization, by maintaining high accuracy and improving generalization capabilities.
  arXiv  Detail & Related papers  (2024-09-11T05:55:51Z)
• TCCT-Net: Two-Stream Network Architecture for Fast and Efficient Engagement Estimation via Behavioral Feature Signals [58.865901821451295]
  We present a novel two-stream feature fusion "Tensor-Convolution and Convolution-Transformer Network" (TCCT-Net) architecture. To better learn the meaningful patterns in the temporal-spatial domain, we design a "CT" stream that integrates a hybrid convolutional-transformer. In parallel, to efficiently extract rich patterns from the temporal-frequency domain, we introduce a "TC" stream that uses Continuous Wavelet Transform (CWT) to represent information in a 2D tensor form.
  arXiv  Detail & Related papers  (2024-04-15T06:01:48Z)
• Heterogenous Memory Augmented Neural Networks [84.29338268789684]
  We introduce a novel heterogeneous memory augmentation approach for neural networks. By introducing learnable memory tokens with attention mechanism, we can effectively boost performance without huge computational overhead. We show our approach on various image and graph-based tasks under both in-distribution (ID) and out-of-distribution (OOD) conditions.
  arXiv  Detail & Related papers  (2023-10-17T01:05:28Z)
• Quantum-inspired event reconstruction with Tensor Networks: Matrix Product States [0.0]
  We show that Networks are ideal vehicles to connect quantum mechanical concepts to machine learning techniques. We show that entanglement entropy can be used to interpret what a network learns.
  arXiv  Detail & Related papers  (2021-06-15T18:00:02Z)
• On Infinite-Width Hypernetworks [101.03630454105621]
  We show that hypernetworks do not guarantee to a global minima under descent. We identify the functional priors of these architectures by deriving their corresponding GP and NTK kernels. As part of this study, we make a mathematical contribution by deriving tight bounds on high order Taylor terms of standard fully connected ReLU networks.
  arXiv  Detail & Related papers  (2020-03-27T00:50:29Z)
• EPINE: Enhanced Proximity Information Network Embedding [2.257737378757467]
  In this work, we devote to mining valuable information in adjacency matrices at a deeper level. Under the same objective, many NE methods calculate high-order proximity by the powers of adjacency matrices. We propose to redefine high-order proximity in a more intuitive manner.
  arXiv  Detail & Related papers  (2020-03-04T15:57:17Z)
• Efficient Structure-preserving Support Tensor Train Machine [0.0]
  Train Multi-way Multi-level Kernel (TT-MMK) We develop the Train Multi-way Multi-level Kernel (TT-MMK), which combines the simplicity of the Polyadic decomposition, the classification power of the Dual Structure-preserving Support Machine, and the reliability of the Train Vector approximation. We show by experiments that the TT-MMK method is usually more reliable, less sensitive to tuning parameters, and gives higher prediction accuracy in the SVM classification when benchmarked against other state-of-the-art techniques.
  arXiv  Detail & Related papers  (2020-02-12T16:35:10Z)
• A Multi-Scale Tensor Network Architecture for Classification and Regression [0.0]
  We present an algorithm for supervised learning using tensor networks. We employ a step of preprocessing the data by coarse-graining through a sequence of wavelet transformations. We show how fine-graining through the network may be used to initialize models with access to finer-scale features.
  arXiv  Detail & Related papers  (2020-01-22T21:26:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.