Sample Efficient Learning of Factored Embeddings of Tensor Fields

• URL: http://arxiv.org/abs/2209.00372v2
• Date: Sat, 9 Mar 2024 03:16:12 GMT
• Title: Sample Efficient Learning of Factored Embeddings of Tensor Fields
• Authors: Taemin Heo, Chandrajit Bajaj
• Abstract summary: We learn approximate full-rank and compact tensor sketches with decompositive representations. All information querying and post-processing on the original tensor field can now be achieved more efficiently.
• Score: 3.0072182643196217
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Data tensors of orders 2 and greater are now routinely being generated. These data collections are increasingly huge and growing. Many scientific and medical data tensors are tensor fields (e.g., images, videos, geographic data) in which the spatial neighborhood contains important information. Directly accessing such large data tensor collections for information has become increasingly prohibitive. We learn approximate full-rank and compact tensor sketches with decompositive representations providing compact space, time and spectral embeddings of tensor fields. All information querying and post-processing on the original tensor field can now be achieved more efficiently and with customizable accuracy as they are performed on these compact factored sketches in latent generative space. We produce optimal rank-r sketchy Tucker decomposition of arbitrary order data tensors by building compact factor matrices from a sample-efficient sub-sampling of tensor slices. Our sample efficient policy is learned via an adaptable stochastic Thompson sampling using Dirichlet distributions with conjugate priors.

Related papers

• Unlocking Data-free Low-bit Quantization with Matrix Decomposition for KV Cache Compression [87.5604418100301]
  Key-value( KV) caching is an important technique to accelerate the inference of large language models. Existing methods often compromise precision or require extra data for calibration. We introduce textbfDecoQuant, a novel data-free low-bit quantization technique based on tensor decomposition methods.
  arXiv  Detail & Related papers  (2024-05-21T08:35:10Z)
• Scalable CP Decomposition for Tensor Learning using GPU Tensor Cores [47.87810316745786]
  We propose a compression-based tensor decomposition framework, namely the exascale-tensor, to support exascale tensor decomposition. Compared to the baselines, the exascale-tensor supports 8,000x larger tensors and a speedup up to 6.95x. We also apply our method to two real-world applications, including gene analysis and tensor layer neural networks.
  arXiv  Detail & Related papers  (2023-11-22T21:04:59Z)
• Multidimensional Data Analysis Based on Block Convolutional Tensor Decomposition [1.1674893622721483]
  We propose a new tensor-tensor product called the $star_ctext-Product$ based on Block convolution with reflective boundary conditions. We also introduce a tensor decomposition based on our $star_ctext-Product$ for arbitrary order tensors. Compared to t-SVD, our new decomposition has lower complexity, and experiments show that it yields higher-quality results in applications such as classification and compression.
  arXiv  Detail & Related papers  (2023-08-03T13:58:21Z)
• Low-Rank Tensor Function Representation for Multi-Dimensional Data Recovery [52.21846313876592]
  Low-rank tensor function representation (LRTFR) can continuously represent data beyond meshgrid with infinite resolution. We develop two fundamental concepts for tensor functions, i.e., the tensor function rank and low-rank tensor function factorization. Our method substantiates the superiority and versatility of our method as compared with state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-12-01T04:00:38Z)
• Influence-guided Data Augmentation for Neural Tensor Completion [21.625908410873944]
  We propose DAIN, a general data augmentation framework that enhances the prediction accuracy of neural tensor completion methods. In this paper, we show that DAIN outperforms all data augmentation baselines in terms of enhancing imputation accuracy of neural tensor completion.
  arXiv  Detail & Related papers  (2021-08-23T15:38:59Z)
• MTC: Multiresolution Tensor Completion from Partial and Coarse Observations [49.931849672492305]
  Existing completion formulation mostly relies on partial observations from a single tensor. We propose an efficient Multi-resolution Completion model (MTC) to solve the problem.
  arXiv  Detail & Related papers  (2021-06-14T02:20:03Z)
• Cherry-Picking Gradients: Learning Low-Rank Embeddings of Visual Data via Differentiable Cross-Approximation [53.95297550117153]
  We propose an end-to-end trainable framework that processes large-scale visual data tensors by looking emphat a fraction of their entries only. The proposed approach is particularly useful for large-scale multidimensional grid data, and for tasks that require context over a large receptive field.
  arXiv  Detail & Related papers  (2021-05-29T08:39:57Z)
• Multi-version Tensor Completion for Time-delayed Spatio-temporal Data [50.762087239885936]
  Real-world-temporal data is often incomplete or inaccurate due to various data loading delays. We propose a low-rank tensor model to predict the updates over time. We obtain up to 27.2% lower root mean-squared-error compared to the best baseline method.
  arXiv  Detail & Related papers  (2021-05-11T19:55:56Z)
• Low-Rank and Sparse Enhanced Tucker Decomposition for Tensor Completion [3.498620439731324]
  We introduce a unified low-rank and sparse enhanced Tucker decomposition model for tensor completion. Our model possesses a sparse regularization term to promote a sparse core tensor, which is beneficial for tensor data compression. It is remarkable that our model is able to deal with different types of real-world data sets, since it exploits the potential periodicity and inherent correlation properties appeared in tensors.
  arXiv  Detail & Related papers  (2020-10-01T12:45:39Z)
• Distributed Non-Negative Tensor Train Decomposition [3.2264685979617655]
  High-dimensional data is presented as multidimensional arrays, aka tensors. The presence of latent (not directly observable) structures in the tensor allows a unique representation and compression of the data. We introduce a distributed non-negative tensor-train and demonstrate its scalability and the compression on synthetic and real-world big datasets.
  arXiv  Detail & Related papers  (2020-08-04T05:35:57Z)
• Spatio-Temporal Tensor Sketching via Adaptive Sampling [15.576219771198389]
  We propose SkeTenSmooth, a novel tensor factorization framework that uses adaptive sampling to compress tensor slices in a temporally streaming fashion. Experiments on the New York City Yellow Taxi data show that SkeTenSmooth greatly reduces the memory cost and random sampling rate.
  arXiv  Detail & Related papers  (2020-06-21T23:55:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.