Structure-Preserving Nonlinear Sufficient Dimension Reduction for Tensors

• URL: http://arxiv.org/abs/2512.20057v1
• Date: Tue, 23 Dec 2025 05:19:16 GMT
• Title: Structure-Preserving Nonlinear Sufficient Dimension Reduction for Tensors
• Authors: Dianjun Lin, Bing Li, Lingzhou Xue,
• Abstract summary: We introduce two nonlinear sufficient dimension reduction methods for regressions with tensor-valued predictors.<n>Our goal is two-fold: the first is to preserve the tensor structure when performing dimension reduction, particularly the meaning of the tensor modes, for improved interpretation.<n>The second is to substantially reduce the number of parameters in dimension reduction, thereby achieving model parsimony and enhancing estimation accuracy.
• Score: 8.679002268074683
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce two nonlinear sufficient dimension reduction methods for regressions with tensor-valued predictors. Our goal is two-fold: the first is to preserve the tensor structure when performing dimension reduction, particularly the meaning of the tensor modes, for improved interpretation; the second is to substantially reduce the number of parameters in dimension reduction, thereby achieving model parsimony and enhancing estimation accuracy. Our two tensor dimension reduction methods echo the two commonly used tensor decomposition mechanisms: one is the Tucker decomposition, which reduces a larger tensor to a smaller one; the other is the CP-decomposition, which represents an arbitrary tensor as a sequence of rank-one tensors. We developed the Fisher consistency of our methods at the population level and established their consistency and convergence rates. Both methods are easy to implement numerically: the Tucker-form can be implemented through a sequence of least-squares steps, and the CP-form can be implemented through a sequence of singular value decompositions. We investigated the finite-sample performance of our methods and showed substantial improvement in accuracy over existing methods in simulations and two data applications.

Related papers

• StoTAM: Stochastic Alternating Minimization for Tucker-Structured Tensor Sensing [7.549565266107219]
  Low-rank tensor sensing is a fundamental problem with broad applications in signal processing and machine learning.<n>Existing recovery methods either operate on the full tensor variable with expensive tensor projections, or adopt factorized formulations that still rely on full-gradient computations.<n>In this work, we propose a alternating minimization algorithm that operates directly on the core tensor and factor matrices under a Tucker factorization.
  arXiv  Detail & Related papers  (2026-01-20T02:18:20Z)
• Efficient Low-Tubal-Rank Tensor Estimation via Alternating Preconditioned Gradient Descent [16.904277250824055]
  Low-tubal-rank tensor estimation is a fundamental task with wide applications across high-dimensional signal processing, machine learning, and image science.<n>Recent approaches address this issue by factorizing the tensor into two smaller factor tensors and solving the resulting problem using gradient descent.<n>We propose an Alternating Preconditioned Gradient Descent (APGD) algorithm, which accelerates convergence in the over- parameterized setting.
  arXiv  Detail & Related papers  (2025-12-08T12:17:40Z)
• Normalized Iterative Hard Thresholding for Tensor Recovery [7.5277782201584085]
  Low-rank recovery builds upon ideas from the theory of compressive sensing.<n>We propose a tensor extension of NIHT, referred to as TNIHT, for the recovery of low-rank tensors.
  arXiv  Detail & Related papers  (2025-07-06T03:36:50Z)
• Score-Based Model for Low-Rank Tensor Recovery [49.158601255093416]
  Low-rank tensor decompositions (TDs) provide an effective framework for multiway data analysis.<n>Traditional TD methods rely on predefined structural assumptions, such as CP or Tucker decompositions.<n>We propose a score-based model that eliminates the need for predefined structural or distributional assumptions.
  arXiv  Detail & Related papers  (2025-06-27T15:05:37Z)
• Low-Rank Tensor Recovery via Variational Schatten-p Quasi-Norm and Jacobian Regularization [49.85875869048434]
  We propose a CP-based low-rank tensor function parameterized by neural networks for implicit neural representation.<n>To achieve sparser CP decomposition, we introduce a variational Schatten-p quasi-norm to prune redundant rank-1 components.<n>For smoothness, we propose a regularization term based on the spectral norm of the Jacobian and Hutchinson's trace estimator.
  arXiv  Detail & Related papers  (2025-06-27T11:23:10Z)
• Low-Rank Tensor Learning by Generalized Nonconvex Regularization [25.115066273660478]
  We study the problem of low-rank tensor learning, where only a few samples are observed the underlying tensor. A family of non tensor learning functions are employed to characterize the low-rankness of the underlying tensor. An algorithm designed to solve the resulting majorization-minimization is proposed.
  arXiv  Detail & Related papers  (2024-10-24T03:33:20Z)
• Error Analysis of Tensor-Train Cross Approximation [88.83467216606778]
  We provide accuracy guarantees in terms of the entire tensor for both exact and noisy measurements. Results are verified by numerical experiments, and may have important implications for the usefulness of cross approximations for high-order tensors.
  arXiv  Detail & Related papers  (2022-07-09T19:33:59Z)
• Scaling and Scalability: Provable Nonconvex Low-Rank Tensor Estimation from Incomplete Measurements [30.395874385570007]
  A fundamental task is to faithfully recover tensors from highly incomplete measurements. We develop an algorithm to directly recover the tensor factors in the Tucker decomposition. We show that it provably converges at a linear independent rate of the ground truth tensor for two canonical problems.
  arXiv  Detail & Related papers  (2021-04-29T17:44:49Z)
• Understanding Implicit Regularization in Over-Parameterized Single Index Model [55.41685740015095]
  We design regularization-free algorithms for the high-dimensional single index model. We provide theoretical guarantees for the induced implicit regularization phenomenon.
  arXiv  Detail & Related papers  (2020-07-16T13:27:47Z)
• Path Sample-Analytic Gradient Estimators for Stochastic Binary Networks [78.76880041670904]
  In neural networks with binary activations and or binary weights the training by gradient descent is complicated. We propose a new method for this estimation problem combining sampling and analytic approximation steps. We experimentally show higher accuracy in gradient estimation and demonstrate a more stable and better performing training in deep convolutional models.
  arXiv  Detail & Related papers  (2020-06-04T21:51:21Z)
• Multi-View Spectral Clustering Tailored Tensor Low-Rank Representation [105.33409035876691]
  This paper explores the problem of multi-view spectral clustering (MVSC) based on tensor low-rank modeling. We design a novel structured tensor low-rank norm tailored to MVSC. We show that the proposed method outperforms state-of-the-art methods to a significant extent.
  arXiv  Detail & Related papers  (2020-04-30T11:52:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.