Learning Doubly Sparse Explicitly Conditioned Transforms

• URL: http://arxiv.org/abs/2606.10975v1
• Date: Tue, 09 Jun 2026 15:13:36 GMT
• Title: Learning Doubly Sparse Explicitly Conditioned Transforms
• Abstract summary: We introduce a structured, explicitly conditioned transform formulated as the product of a fixed canonical matrix and a data-adaptive sparse component.<n>This approach seeks to preserve the advantages of fast and stable analytical transforms, while introducing controllable adaptivity to the data.<n> Empirical results demonstrate state-of-the-art results on the doubly sparse transform learning problem and comparable performance with its dense variant.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Finding convenient spaces in which certain hypotheses regarding an assumed sparse structure of natural signals hold true has become a desirable result in recent research, its implications being reflected in areas such as data compression, noise reduction and feature extraction. While the extensively used analytical transforms, such as DFT or DCT, already provide efficient algorithms and robust sparse representations, they assume a fixed prior about the data, failing to accurately capture the specific structure of more restrictive classes of signals. To address this, the concept of a data-adaptive, learnt transform has been introduced in the literature, allowing for the reduction of a residual term in the transform domain. More recent studies have shown that the condition number serves as a good metric in this context, where the desired outcome alternates between a generalizing tendency and one that achieves minimal approximation error. Motivated by these considerations, we introduce the learning of a structured, explicitly conditioned transform formulated as the product of a fixed canonical matrix and a refining data-adaptive sparse component. This approach seeks to preserve the advantages of fast and stable analytical transforms, while introducing controllable adaptivity to the data. No references that concern this specific formulation have been identified so far, indicating its novelty. The proposed algorithm is motivated within the framework of inexact proximal methods, leveraging a newly derived closed-form projection operator. Empirical observations demonstrate state-of-the-art results on the doubly sparse transform learning problem and comparable performance with its dense variant at significantly lower computational costs and sometimes faster convergence and better avoidance of bad local minima.

Related papers

• Semi-Supervised Conditional Generative Learning through Stochastic Interpolation and Sufficient Representations [12.299177496689621]
  We propose a semi-supervised framework combining conditional intrinsic with low-dimensional latent representations.<n>RepG decomposes generation into two stages: label-dependent latent sampling and high-dimensional reconstruction.
  arXiv  Detail & Related papers  (2026-07-18T09:21:53Z)
• Rethinking Structural Anomaly Detection: From Decision Boundaries to Projection Operators [51.771100367755544]
  We introduce a geometric perspective to address the mismatch between the inductive bias of existing methods and the structure of the data.<n>Specifically, we learn a projection operator onto the manifold of normal samples and define a sample as anomalous if it is altered by this projection.<n> Notably, it provides a unifying interpretation of reconstruction-based methods by explaining their success and failure in terms of projection quality.
  arXiv  Detail & Related papers  (2026-06-13T12:37:20Z)
• Variational Rectification Inference for Learning with Noisy Labels [74.85528327499662]
  We propose variational rectification inference (VRI) to formulate the adaptive rectification for loss functions.<n>VRI is constructed as a hierarchical Bayes by treating the rectifying vector as a latent variable.<n>By introducing the variational term in VRI, the conditional posterior is estimated accurately and avoids collapsing to a Dirac delta function.
  arXiv  Detail & Related papers  (2026-03-18T01:25:08Z)
• From Invariant Representations to Invariant Data: Provable Robustness to Spurious Correlations via Noisy Counterfactual Matching [14.679039741833305]
  Models that learn spurious correlations from training data often fail when deployed in new environments.<n>We propose a data-centric alternative that shifts the focus from learning invariant representations to leveraging invariant data pairs.<n>Noisy Counterfactual Matching improves robustness by leveraging even a small number of emphnoisy counterfactual pairs.
  arXiv  Detail & Related papers  (2025-05-30T17:42:32Z)
• PseudoNeg-MAE: Self-Supervised Point Cloud Learning using Conditional Pseudo-Negative Embeddings [55.55445978692678]
  PseudoNeg-MAE enhances global feature representation of point cloud masked autoencoders by making them both discriminative and sensitive to transformations.<n>We propose a novel loss that explicitly penalizes invariant collapse, enabling the network to capture richer transformation cues while preserving discriminative representations.
  arXiv  Detail & Related papers  (2024-09-24T07:57:21Z)
• Enabling Tensor Decomposition for Time-Series Classification via A Simple Pseudo-Laplacian Contrast [26.28414569796961]
  We propose a novel Pseudo Laplacian Contrast (PLC) tensor decomposition framework. It integrates the data augmentation and cross-view Laplacian to enable the extraction of class-aware representations. Experiments on various datasets demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2024-09-23T16:48:13Z)
• Learning Explicitly Conditioned Sparsifying Transforms [7.335712499936904]
  We consider a new sparsifying transform model that enforces explicit control over the data representation quality and the condition number of the learned transforms. We confirm through numerical experiments that our model presents better numerical behavior than the state-of-the-art.
  arXiv  Detail & Related papers  (2024-03-05T18:03:51Z)
• Uncovering mesa-optimization algorithms in Transformers [61.06055590704677]
  Some autoregressive models can learn as an input sequence is processed, without undergoing any parameter changes, and without being explicitly trained to do so. We show that standard next-token prediction error minimization gives rise to a subsidiary learning algorithm that adjusts the model as new inputs are revealed. Our findings explain in-context learning as a product of autoregressive loss minimization and inform the design of new optimization-based Transformer layers.
  arXiv  Detail & Related papers  (2023-09-11T22:42:50Z)
• Learning Unnormalized Statistical Models via Compositional Optimization [73.30514599338407]
  Noise-contrastive estimation(NCE) has been proposed by formulating the objective as the logistic loss of the real data and the artificial noise. In this paper, we study it a direct approach for optimizing the negative log-likelihood of unnormalized models.
  arXiv  Detail & Related papers  (2023-06-13T01:18:16Z)
• Score-based Causal Representation Learning with Interventions [54.735484409244386]
  This paper studies the causal representation learning problem when latent causal variables are observed indirectly. The objectives are: (i) recovering the unknown linear transformation (up to scaling) and (ii) determining the directed acyclic graph (DAG) underlying the latent variables.
  arXiv  Detail & Related papers  (2023-01-19T18:39:48Z)
• Which Invariance Should We Transfer? A Causal Minimax Learning Approach [18.71316951734806]
  We present a comprehensive minimax analysis from a causal perspective. We propose an efficient algorithm to search for the subset with minimal worst-case risk. The effectiveness and efficiency of our methods are demonstrated on synthetic data and the diagnosis of Alzheimer's disease.
  arXiv  Detail & Related papers  (2021-07-05T09:07:29Z)
• Recursive Inference for Variational Autoencoders [34.552283758419506]
  Inference networks of traditional Variational Autoencoders (VAEs) are typically amortized. Recent semi-amortized approaches were proposed to address this drawback. We introduce an accurate amortized inference algorithm.
  arXiv  Detail & Related papers  (2020-11-17T10:22:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.