Kernel Regression with Tensor Trains and Hadamard Overparameterization

• URL: http://arxiv.org/abs/2607.17390v2
• Date: Tue, 21 Jul 2026 12:57:11 GMT
• Title: Kernel Regression with Tensor Trains and Hadamard Overparameterization
• Abstract summary: KReTTaH is introduced as a training-data-free, interpretable, and nonparametric framework for multi-way data imputation.<n>It consistently outperforms state-of-the-art tensor-, Bayesian-, and neural-network-based baselines in terms of modeling accuracy.
• Score: 9.941965164307843
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Kernel regression with tensor trains and Hadamard overparameterization (KReTTaH) is introduced as a training-data-free, interpretable, and nonparametric framework for multi-way data imputation. The imputation problem is reformulated as regression in reproducing kernel Hilbert spaces (RKHS), where the tensor regression coefficients are explicitly constrained to lie on fixed-rank tensor-train (TT) manifolds and structured via Hadamard overparameterization to promote sparsity and high representational efficiency. Rather than relying on costly cross-validation, KReTTaH jointly optimizes the TT coefficient tensors and the kernel covariance matrices within a Riemannian product-manifold framework -- the former on fixed-rank TT manifolds, the latter on the manifold of positive-definite matrices -- thereby enabling automated kernel-hyperparameter selection. Numerical tests on two challenging applications -- imputation of high-dimensional functional magnetic resonance imaging (fMRI) data and recovery of missing edge flows in dynamic graphs -- demonstrate that KReTTaH consistently outperforms state-of-the-art tensor-, Bayesian-, and neural-network-based baselines in terms of modeling accuracy.

Related papers

• Guaranteed Noisy CP Tensor Recovery via Riemannian Optimization on the Segre Manifold [9.804487437104289]
  We exploit the intrinsic geometry of rank-one tensors by casting the recovery task as an optimization problem over the Segre manifold.<n>We prove that RGD converges at a local linear rate, while RGN exhibits an initial local quadratic convergence phase that transitions to a linear rate as the iterates approach the statistical noise floor.
  arXiv  Detail & Related papers  (2025-10-01T06:44:52Z)
• Kernel Regression of Multi-Way Data via Tensor Trains with Hadamard Overparametrization: The Dynamic Graph Flow Case [9.941965164307843]
  Kernel Regression via Trains with Hadamard overparametrization (KReTTaH) is a regression-based framework for interpretable multi-way data imputation.<n>KReTTaH consistently outperforms state-of-the-art alternatives.
  arXiv  Detail & Related papers  (2025-09-26T11:00:05Z)
• Generalized Nonnegative Structured Kruskal Tensor Regression [22.300007523556022]
  Generalized Nonnegative Structured Kruskal Regression (NS-KTR) is a novel tensor regression framework.<n>It enhances interpretability and performance through mode-specific hybrid regularization and nonnegativity constraints.
  arXiv  Detail & Related papers  (2025-09-24T08:51:38Z)
• Graph-based Clustering Revisited: A Relaxation of Kernel $k$-Means Perspective [73.18641268511318]
  We propose a graph-based clustering algorithm that only relaxes the orthonormal constraint to derive clustering results.<n>To ensure a doubly constraint into a gradient, we transform the non-negative constraint into a class probability parameter.
  arXiv  Detail & Related papers  (2025-09-23T09:14:39Z)
• Low-Rank Tensor Recovery via Variational Schatten-p Quasi-Norm and Jacobian Regularization [49.85875869048434]
  We propose a CP-based low-rank tensor function parameterized by neural networks for implicit neural representation.<n>To achieve sparser CP decomposition, we introduce a variational Schatten-p quasi-norm to prune redundant rank-1 components.<n>For smoothness, we propose a regularization term based on the spectral norm of the Jacobian and Hutchinson's trace estimator.
  arXiv  Detail & Related papers  (2025-06-27T11:23:10Z)
• Generalization Bound of Gradient Flow through Training Trajectory and Data-dependent Kernel [55.82768375605861]
  We establish a generalization bound for gradient flow that aligns with the classical Rademacher complexity for kernel methods.<n>Unlike static kernels such as NTK, the LPK captures the entire training trajectory, adapting to both data and optimization dynamics.
  arXiv  Detail & Related papers  (2025-06-12T23:17:09Z)
• TensorGRaD: Tensor Gradient Robust Decomposition for Memory-Efficient Neural Operator Training [91.8932638236073]
  We introduce textbfTensorGRaD, a novel method that directly addresses the memory challenges associated with large-structured weights.<n>We show that sparseGRaD reduces total memory usage by over $50%$ while maintaining and sometimes even improving accuracy.
  arXiv  Detail & Related papers  (2025-01-04T20:51:51Z)
• Tensor-on-Tensor Regression: Riemannian Optimization, Over-parameterization, Statistical-computational Gap, and Their Interplay [9.427635404752936]
  We study the tensor-on-tensor regression, where the goal is to connect tensor responses to tensor covariates with a low Tucker rank parameter/matrix. We propose two methods to cope with the challenge of unknown rank. We provide the first convergence guarantee for the general tensor-on-tensor regression.
  arXiv  Detail & Related papers  (2022-06-17T13:15:27Z)
• Scalable Spatiotemporally Varying Coefficient Modelling with Bayesian Kernelized Tensor Regression [17.158289775348063]
  Kernelized tensor Regression (BKTR) can be considered a new and scalable approach to modeling processes with low-rank cotemporal structure. We conduct extensive experiments on both synthetic and real-world data sets, and our results confirm the superior performance and efficiency of BKTR for model estimation and inference.
  arXiv  Detail & Related papers  (2021-08-31T19:22:23Z)
• Spectral Tensor Train Parameterization of Deep Learning Layers [136.4761580842396]
  We study low-rank parameterizations of weight matrices with embedded spectral properties in the Deep Learning context. We show the effects of neural network compression in the classification setting and both compression and improved stability training in the generative adversarial training setting.
  arXiv  Detail & Related papers  (2021-03-07T00:15:44Z)
• Multi-View Spectral Clustering Tailored Tensor Low-Rank Representation [105.33409035876691]
  This paper explores the problem of multi-view spectral clustering (MVSC) based on tensor low-rank modeling. We design a novel structured tensor low-rank norm tailored to MVSC. We show that the proposed method outperforms state-of-the-art methods to a significant extent.
  arXiv  Detail & Related papers  (2020-04-30T11:52:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.