Tensor Decomposition with Unaligned Observations

• URL: http://arxiv.org/abs/2410.14046v1
• Date: Thu, 17 Oct 2024 21:39:18 GMT
• Title: Tensor Decomposition with Unaligned Observations
• Authors: Runshi Tang, Tamara Kolda, Anru R. Zhang,
• Abstract summary: The mode with unaligned observations is represented using functions in a reproducing kernel Hilbert space (RKHS) We introduce a versatile loss function that effectively accounts for various types of data, including binary, integer-valued, and positive-valued types. A sketching algorithm is also introduced to further improve efficiency when using the $ell$ loss function.
• Score: 4.970364068620608
• License:
• Abstract: This paper presents a canonical polyadic (CP) tensor decomposition that addresses unaligned observations. The mode with unaligned observations is represented using functions in a reproducing kernel Hilbert space (RKHS). We introduce a versatile loss function that effectively accounts for various types of data, including binary, integer-valued, and positive-valued types. Additionally, we propose an optimization algorithm for computing tensor decompositions with unaligned observations, along with a stochastic gradient method to enhance computational efficiency. A sketching algorithm is also introduced to further improve efficiency when using the $\ell_2$ loss function. To demonstrate the efficacy of our methods, we provide illustrative examples using both synthetic data and an early childhood human microbiome dataset.

Related papers

• A Structure-Preserving Kernel Method for Learning Hamiltonian Systems [3.594638299627404]
  A structure-preserving kernel ridge regression method is presented that allows the recovery of potentially high-dimensional and nonlinear Hamiltonian functions. The paper extends kernel regression methods to problems in which loss functions involving linear functions of gradients are required. A full error analysis is conducted that provides convergence rates using fixed and adaptive regularization parameters.
  arXiv  Detail & Related papers  (2024-03-15T07:20:21Z)
• Stochastic Optimization for Non-convex Problem with Inexact Hessian Matrix, Gradient, and Function [99.31457740916815]
  Trust-region (TR) and adaptive regularization using cubics have proven to have some very appealing theoretical properties. We show that TR and ARC methods can simultaneously provide inexact computations of the Hessian, gradient, and function values.
  arXiv  Detail & Related papers  (2023-10-18T10:29:58Z)
• Learning Unnormalized Statistical Models via Compositional Optimization [73.30514599338407]
  Noise-contrastive estimation(NCE) has been proposed by formulating the objective as the logistic loss of the real data and the artificial noise. In this paper, we study it a direct approach for optimizing the negative log-likelihood of unnormalized models.
  arXiv  Detail & Related papers  (2023-06-13T01:18:16Z)
• Score-based Diffusion Models in Function Space [140.792362459734]
  Diffusion models have recently emerged as a powerful framework for generative modeling. We introduce a mathematically rigorous framework called Denoising Diffusion Operators (DDOs) for training diffusion models in function space. We show that the corresponding discretized algorithm generates accurate samples at a fixed cost independent of the data resolution.
  arXiv  Detail & Related papers  (2023-02-14T23:50:53Z)
• Rigorous dynamical mean field theory for stochastic gradient descent methods [17.90683687731009]
  We prove closed-form equations for the exact high-dimensionals of a family of first order gradient-based methods. This includes widely used algorithms such as gradient descent (SGD) or Nesterov acceleration.
  arXiv  Detail & Related papers  (2022-10-12T21:10:55Z)
• On the Benefits of Large Learning Rates for Kernel Methods [110.03020563291788]
  We show that a phenomenon can be precisely characterized in the context of kernel methods. We consider the minimization of a quadratic objective in a separable Hilbert space, and show that with early stopping, the choice of learning rate influences the spectral decomposition of the obtained solution.
  arXiv  Detail & Related papers  (2022-02-28T13:01:04Z)
• Efficient Multidimensional Functional Data Analysis Using Marginal Product Basis Systems [2.4554686192257424]
  We propose a framework for learning continuous representations from a sample of multidimensional functional data. We show that the resulting estimation problem can be solved efficiently by the tensor decomposition. We conclude with a real data application in neuroimaging.
  arXiv  Detail & Related papers  (2021-07-30T16:02:15Z)
• Optimal oracle inequalities for solving projected fixed-point equations [53.31620399640334]
  We study methods that use a collection of random observations to compute approximate solutions by searching over a known low-dimensional subspace of the Hilbert space. We show how our results precisely characterize the error of a class of temporal difference learning methods for the policy evaluation problem with linear function approximation.
  arXiv  Detail & Related papers  (2020-12-09T20:19:32Z)
• SGB: Stochastic Gradient Bound Method for Optimizing Partition Functions [15.33098084159285]
  This paper addresses the problem of optimizing partition functions in a learning setting. We propose a variant of the bound majorization algorithm that relies on upper-bounding the partition function with a quadratic surrogate.
  arXiv  Detail & Related papers  (2020-11-03T04:42:51Z)
• SLEIPNIR: Deterministic and Provably Accurate Feature Expansion for Gaussian Process Regression with Derivatives [86.01677297601624]
  We propose a novel approach for scaling GP regression with derivatives based on quadrature Fourier features. We prove deterministic, non-asymptotic and exponentially fast decaying error bounds which apply for both the approximated kernel as well as the approximated posterior.
  arXiv  Detail & Related papers  (2020-03-05T14:33:20Z)
• Nonlinear Functional Output Regression: a Dictionary Approach [1.8160945635344528]
  We introduce projection learning (PL), a novel dictionary-based approach that learns to predict a function that is expanded on a dictionary. PL minimizes an empirical risk based on a functional loss. PL enjoys a particularily attractive trade-off between computational cost and performances.
  arXiv  Detail & Related papers  (2020-03-03T10:31:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.