Low-Rank Tensor Recovery via Variational Schatten-p Quasi-Norm and Jacobian Regularization

• URL: http://arxiv.org/abs/2506.22134v2
• Date: Tue, 07 Oct 2025 14:29:44 GMT
• Title: Low-Rank Tensor Recovery via Variational Schatten-p Quasi-Norm and Jacobian Regularization
• Authors: Zhengyun Cheng, Ruizhe Zhang, Guanwen Zhang, Yi Xu, Xiangyang Ji, Wei Zhou,
• Abstract summary: We propose a CP-based low-rank tensor function parameterized by neural networks for implicit neural representation.<n>To achieve sparser CP decomposition, we introduce a variational Schatten-p quasi-norm to prune redundant rank-1 components.<n>For smoothness, we propose a regularization term based on the spectral norm of the Jacobian and Hutchinson's trace estimator.
• Score: 49.85875869048434
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Higher-order tensors are well-suited for representing multi-dimensional data, such as images and videos, which typically characterize low-rank structures. Low-rank tensor decomposition has become essential in machine learning and computer vision, but existing methods like Tucker decomposition offer flexibility at the expense of interpretability. The CANDECOMP/PARAFAC (CP) decomposition provides a natural and interpretable structure, while obtaining a sparse solutions remains challenging. Leveraging the rich properties of CP decomposition, we propose a CP-based low-rank tensor function parameterized by neural networks (NN) for implicit neural representation. This approach can model the tensor both on-grid and beyond grid, fully utilizing the non-linearity of NN with theoretical guarantees on excess risk bounds. To achieve sparser CP decomposition, we introduce a variational Schatten-p quasi-norm to prune redundant rank-1 components and prove that it serves as a common upper bound for the Schatten-p quasi-norms of arbitrary unfolding matrices. For smoothness, we propose a regularization term based on the spectral norm of the Jacobian and Hutchinson's trace estimator. The proposed smoothness regularization is SVD-free and avoids explicit chain rule derivations. It can serve as an alternative to Total Variation (TV) regularization in image denoising tasks and is naturally applicable to implicit neural representation. Extensive experiments on multi-dimensional data recovery tasks, including image inpainting, denoising, and point cloud upsampling, demonstrate the superiority and versatility of our method compared to state-of-the-art approaches. The code is available at https://github.com/CZY-Code/CP-Pruner.

Related papers

• SDTN and TRN: Adaptive Spectral-Spatial Feature Extraction for Hyperspectral Image Classification [1.2871580250533408]
  Hyperspectral image classification plays a pivotal role in precision agriculture, providing accurate insights into crop health monitoring, disease detection, and soil analysis.<n>Traditional methods struggle with high-dimensional data, spectral-spatial redundancy, and the scarcity of labeled samples, often leading to suboptimal performance.<n>To address these challenges, we propose the Self-Adaptive- Regularized Network (SDTN), which combines tensor decomposition with regularization mechanisms to dynamically adjust tensor ranks.<n>This approach not only maintains high classification accuracy but also significantly reduces computational complexity, making the framework highly suitable for real-time deployment in resource-constrained environments.
  arXiv  Detail & Related papers  (2025-07-13T04:53:33Z)
• Score-Based Model for Low-Rank Tensor Recovery [49.158601255093416]
  Low-rank tensor decompositions (TDs) provide an effective framework for multiway data analysis.<n>Traditional TD methods rely on predefined structural assumptions, such as CP or Tucker decompositions.<n>We propose a score-based model that eliminates the need for predefined structural or distributional assumptions.
  arXiv  Detail & Related papers  (2025-06-27T15:05:37Z)
• sparseGeoHOPCA: A Geometric Solution to Sparse Higher-Order PCA Without Covariance Estimation [8.802387139798808]
  We propose a novel framework for high-order principal component analysis (SHOPCA) that introduces a perspective to tensor-dependent decomposition.<n>We show that sparseGeoHOP supports in high-dimensional image settings and on ImageNet.
  arXiv  Detail & Related papers  (2025-06-10T10:30:48Z)
• TensorGRaD: Tensor Gradient Robust Decomposition for Memory-Efficient Neural Operator Training [91.8932638236073]
  We introduce textbfTensorGRaD, a novel method that directly addresses the memory challenges associated with large-structured weights.<n>We show that sparseGRaD reduces total memory usage by over $50%$ while maintaining and sometimes even improving accuracy.
  arXiv  Detail & Related papers  (2025-01-04T20:51:51Z)
• tCURLoRA: Tensor CUR Decomposition Based Low-Rank Parameter Adaptation and Its Application in Medical Image Segmentation [1.3281936946796913]
  Transfer learning, by leveraging knowledge from pre-trained models, has significantly enhanced the performance of target tasks.<n>As deep neural networks scale up, full fine-tuning introduces substantial computational and storage challenges.<n>We propose tCURLoRA, a novel fine-tuning method based on tensor CUR decomposition.
  arXiv  Detail & Related papers  (2025-01-04T08:25:32Z)
• OTLRM: Orthogonal Learning-based Low-Rank Metric for Multi-Dimensional Inverse Problems [14.893020063373022]
  We introduce a novel data-driven generative low-rank t-SVD model based on the learnable orthogonal transform.<n>We also propose a low-rank solver as a generalization of SVT, which utilizes an efficient representation of generative networks to obtain low-rank structures.
  arXiv  Detail & Related papers  (2024-12-15T12:28:57Z)
• Irregular Tensor Low-Rank Representation for Hyperspectral Image Representation [71.69331824668954]
  Spectral variations pose a common challenge in analyzing hyperspectral images (HSI)<n>Low-rank tensor representation has emerged as a robust strategy, leveraging inherent correlations within HSI data.<n>We propose a novel model for irregular tensor lowrank representation tailored to efficiently model irregular 3D cubes.
  arXiv  Detail & Related papers  (2024-10-24T02:56:22Z)
• Enabling Tensor Decomposition for Time-Series Classification via A Simple Pseudo-Laplacian Contrast [26.28414569796961]
  We propose a novel Pseudo Laplacian Contrast (PLC) tensor decomposition framework. It integrates the data augmentation and cross-view Laplacian to enable the extraction of class-aware representations. Experiments on various datasets demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2024-09-23T16:48:13Z)
• Convolutional Neural Network Compression Based on Low-Rank Decomposition [3.3295360710329738]
  This paper proposes a model compression method that integrates Variational Bayesian Matrix Factorization. VBMF is employed to estimate the rank of the weight tensor at each layer. Experimental results show that for both high and low compression ratios, our compression model exhibits advanced performance.
  arXiv  Detail & Related papers  (2024-08-29T06:40:34Z)
• The Convex Landscape of Neural Networks: Characterizing Global Optima and Stationary Points via Lasso Models [75.33431791218302]
  Deep Neural Network Network (DNN) models are used for programming purposes. In this paper we examine the use of convex neural recovery models. We show that all the stationary non-dimensional objective objective can be characterized as the standard a global subsampled convex solvers program. We also show that all the stationary non-dimensional objective objective can be characterized as the standard a global subsampled convex solvers program.
  arXiv  Detail & Related papers  (2023-12-19T23:04:56Z)
• Low-Rank Tensor Function Representation for Multi-Dimensional Data Recovery [52.21846313876592]
  Low-rank tensor function representation (LRTFR) can continuously represent data beyond meshgrid with infinite resolution. We develop two fundamental concepts for tensor functions, i.e., the tensor function rank and low-rank tensor function factorization. Our method substantiates the superiority and versatility of our method as compared with state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-12-01T04:00:38Z)
• Low-Rank and Sparse Enhanced Tucker Decomposition for Tensor Completion [3.498620439731324]
  We introduce a unified low-rank and sparse enhanced Tucker decomposition model for tensor completion. Our model possesses a sparse regularization term to promote a sparse core tensor, which is beneficial for tensor data compression. It is remarkable that our model is able to deal with different types of real-world data sets, since it exploits the potential periodicity and inherent correlation properties appeared in tensors.
  arXiv  Detail & Related papers  (2020-10-01T12:45:39Z)
• Stable Low-rank Tensor Decomposition for Compression of Convolutional Neural Network [19.717842489217684]
  This paper is the first study on degeneracy in the tensor decomposition of convolutional kernels. We present a novel method, which can stabilize the low-rank approximation of convolutional kernels and ensure efficient compression. We evaluate our approach on popular CNN architectures for image classification and show that our method results in much lower accuracy degradation and provides consistent performance.
  arXiv  Detail & Related papers  (2020-08-12T17:10:12Z)
• Multi-View Spectral Clustering Tailored Tensor Low-Rank Representation [105.33409035876691]
  This paper explores the problem of multi-view spectral clustering (MVSC) based on tensor low-rank modeling. We design a novel structured tensor low-rank norm tailored to MVSC. We show that the proposed method outperforms state-of-the-art methods to a significant extent.
  arXiv  Detail & Related papers  (2020-04-30T11:52:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.