Tensor Decomposition Meets RKHS: Efficient Algorithms for Smooth and Misaligned Data

• URL: http://arxiv.org/abs/2408.05677v1
• Date: Sun, 11 Aug 2024 02:58:28 GMT
• Title: Tensor Decomposition Meets RKHS: Efficient Algorithms for Smooth and Misaligned Data
• Abstract summary: We decompose a multidimensional data array into a sum of outer products of finite-dimensional vectors. We refer to tensors with some infinite-dimensional modes as quasitensors. The approach of decomposing a tensor with some continuous RKHS modes is referred to as CP-HiFi.
• Score: 10.005398712684244
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The canonical polyadic (CP) tensor decomposition decomposes a multidimensional data array into a sum of outer products of finite-dimensional vectors. Instead, we can replace some or all of the vectors with continuous functions (infinite-dimensional vectors) from a reproducing kernel Hilbert space (RKHS). We refer to tensors with some infinite-dimensional modes as quasitensors, and the approach of decomposing a tensor with some continuous RKHS modes is referred to as CP-HiFi (hybrid infinite and finite dimensional) tensor decomposition. An advantage of CP-HiFi is that it can enforce smoothness in the infinite dimensional modes. Further, CP-HiFi does not require the observed data to lie on a regular and finite rectangular grid and naturally incorporates misaligned data. We detail the methodology and illustrate it on a synthetic example.

Related papers

• Generalized Canonical Polyadic Tensor Decompositions with General Symmetry [1.9766522384767224]
  Generalized CP (GCP) decompositions generalize this approach by allowing general loss functions.<n>We develop a symmetric GCP (SymGCP) decomposition that allows for general forms of symmetry.<n>We demonstrate the utility of the proposed SymGCP algorithms with a variety of experiments on both synthetic and real data.
  arXiv  Detail & Related papers  (2026-01-08T19:27:52Z)
• Low-Rank Implicit Neural Representation via Schatten-p Quasi-Norm and Jacobian Regularization [49.158601255093416]
  We propose a CP-based low-rank tensor function parameterized by neural networks for implicit neural representation.<n>For smoothness, we propose a regularization term based on the spectral norm of the Jacobian and Hutchinson's trace estimator.<n>Our proposed smoothness regularization is SVD-free and avoids explicit chain rule derivations.
  arXiv  Detail & Related papers  (2025-06-27T11:23:10Z)
• Proper Latent Decomposition [4.266376725904727]
  We compute a reduced set of intrinsic coordinates (latent space) to accurately describe a flow with fewer degrees of freedom than the numerical discretization. With this proposed numerical framework, we propose an algorithm to perform PLD on the manifold. This work opens opportunities for analyzing autoencoders and latent spaces, nonlinear reduced-order modeling and scientific insights into the structure of high-dimensional data.
  arXiv  Detail & Related papers  (2024-12-01T12:19:08Z)
• Highly Adaptive Ridge [84.38107748875144]
  We propose a regression method that achieves a $n-2/3$ dimension-free L2 convergence rate in the class of right-continuous functions with square-integrable sectional derivatives. Har is exactly kernel ridge regression with a specific data-adaptive kernel based on a saturated zero-order tensor-product spline basis expansion. We demonstrate empirical performance better than state-of-the-art algorithms for small datasets in particular.
  arXiv  Detail & Related papers  (2024-10-03T17:06:06Z)
• Learning with Norm Constrained, Over-parameterized, Two-layer Neural Networks [54.177130905659155]
  Recent studies show that a reproducing kernel Hilbert space (RKHS) is not a suitable space to model functions by neural networks. In this paper, we study a suitable function space for over- parameterized two-layer neural networks with bounded norms.
  arXiv  Detail & Related papers  (2024-04-29T15:04:07Z)
• Spectral Norm of Convolutional Layers with Circular and Zero Paddings [55.233197272316275]
  We generalize the use of the Gram iteration to zero padding convolutional layers and prove its quadratic convergence. We also provide theorems for bridging the gap between circular and zero padding convolution's spectral norm.
  arXiv  Detail & Related papers  (2024-01-31T23:48:48Z)
• Functional Bayesian Tucker Decomposition for Continuous-indexed Tensor Data [32.19122007191261]
  We propose Functional Bayesian Tucker Decomposition (FunBaT) to handle multi-aspect data. We use continuous-indexed data as the interaction between the Tucker core and a group of latent functions. An efficient inference algorithm is developed for posterior scalable approximation based on advanced message-passing techniques.
  arXiv  Detail & Related papers  (2023-11-08T16:54:23Z)
• Multidimensional Data Analysis Based on Block Convolutional Tensor Decomposition [1.1674893622721483]
  We propose a new tensor-tensor product called the $star_ctext-Product$ based on Block convolution with reflective boundary conditions. We also introduce a tensor decomposition based on our $star_ctext-Product$ for arbitrary order tensors. Compared to t-SVD, our new decomposition has lower complexity, and experiments show that it yields higher-quality results in applications such as classification and compression.
  arXiv  Detail & Related papers  (2023-08-03T13:58:21Z)
• Low-Rank Tensor Function Representation for Multi-Dimensional Data Recovery [52.21846313876592]
  Low-rank tensor function representation (LRTFR) can continuously represent data beyond meshgrid with infinite resolution. We develop two fundamental concepts for tensor functions, i.e., the tensor function rank and low-rank tensor function factorization. Our method substantiates the superiority and versatility of our method as compared with state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-12-01T04:00:38Z)
• Average-Case Complexity of Tensor Decomposition for Low-Degree Polynomials [93.59919600451487]
  "Statistical-computational gaps" occur in many statistical inference tasks. We consider a model for random order-3 decomposition where one component is slightly larger in norm than the rest. We show that tensor entries can accurately estimate the largest component when $ll n3/2$ but fail to do so when $rgg n3/2$.
  arXiv  Detail & Related papers  (2022-11-10T00:40:37Z)
• 2D+3D facial expression recognition via embedded tensor manifold regularization [16.98176664818354]
  A novel approach via embedded tensor manifold regularization for 2D+3D facial expression recognition (FERETMR) is proposed. We establish the first-order optimality condition in terms of stationary points, and then design a block coordinate descent (BCD) algorithm with convergence analysis. Numerical results on BU-3DFE database and Bosphorus databases demonstrate the effectiveness of our proposed approach.
  arXiv  Detail & Related papers  (2022-01-29T06:11:00Z)
• Distributed Non-Negative Tensor Train Decomposition [3.2264685979617655]
  High-dimensional data is presented as multidimensional arrays, aka tensors. The presence of latent (not directly observable) structures in the tensor allows a unique representation and compression of the data. We introduce a distributed non-negative tensor-train and demonstrate its scalability and the compression on synthetic and real-world big datasets.
  arXiv  Detail & Related papers  (2020-08-04T05:35:57Z)
• Spectral Learning on Matrices and Tensors [74.88243719463053]
  We show that tensor decomposition can pick up latent effects that are missed by matrix methods. We also outline computational techniques to design efficient tensor decomposition methods.
  arXiv  Detail & Related papers  (2020-04-16T22:53:00Z)
• On Recoverability of Randomly Compressed Tensors with Low CP Rank [29.00634848772122]
  We show that if the number of measurements is on the same order of magnitude as that of the model parameters, then the tensor is recoverable. Our proof is based on deriving a textitrestricted isometry property (R.I.P.) under the CPD model via set covering techniques.
  arXiv  Detail & Related papers  (2020-01-08T04:44:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.