Variational Learning ISTA

• URL: http://arxiv.org/abs/2407.06646v1
• Date: Tue, 9 Jul 2024 08:17:06 GMT
• Title: Variational Learning ISTA
• Authors: Fabio Valerio Massoli, Christos Louizos, Arash Behboodi,
• Abstract summary: We propose an architecture for learning sparse representations and reconstructions under varying sensing matrix conditions. We learn a distribution over dictionaries via a variational approach, dubbed Variational Learning ISTA (VLISTA) As a result, VLISTA provides a probabilistic way to jointly learn the dictionary distribution and the reconstruction algorithm with varying sensing matrices.
• Score: 13.894911545678635
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Compressed sensing combines the power of convex optimization techniques with a sparsity-inducing prior on the signal space to solve an underdetermined system of equations. For many problems, the sparsifying dictionary is not directly given, nor its existence can be assumed. Besides, the sensing matrix can change across different scenarios. Addressing these issues requires solving a sparse representation learning problem, namely dictionary learning, taking into account the epistemic uncertainty of the learned dictionaries and, finally, jointly learning sparse representations and reconstructions under varying sensing matrix conditions. We address both concerns by proposing a variant of the LISTA architecture. First, we introduce Augmented Dictionary Learning ISTA (A-DLISTA), which incorporates an augmentation module to adapt parameters to the current measurement setup. Then, we propose to learn a distribution over dictionaries via a variational approach, dubbed Variational Learning ISTA (VLISTA). VLISTA exploits A-DLISTA as the likelihood model and approximates a posterior distribution over the dictionaries as part of an unfolded LISTA-based recovery algorithm. As a result, VLISTA provides a probabilistic way to jointly learn the dictionary distribution and the reconstruction algorithm with varying sensing matrices. We provide theoretical and experimental support for our architecture and show that our model learns calibrated uncertainties.

Related papers

• Unified Lexical Representation for Interpretable Visual-Language Alignment [52.059812317944434]
  We introduce LexVLA, a framework for learning a unified lexical representation for both modalities without complex design. We use DINOv2 as our visual model for its local-inclined features and Llama 2, a generative language model, to leverage its in-context lexical prediction ability. We demonstrate that these two pre-trained uni-modal models can be well-aligned by fine-tuning on the modest multi-modal dataset.
  arXiv  Detail & Related papers  (2024-07-25T07:35:27Z)
• Image2Sentence based Asymmetrical Zero-shot Composed Image Retrieval [92.13664084464514]
  The task of composed image retrieval (CIR) aims to retrieve images based on the query image and the text describing the users' intent. Existing methods have made great progress with the advanced large vision-language (VL) model in CIR task, however, they generally suffer from two main issues: lack of labeled triplets for model training and difficulty of deployment on resource-restricted environments. We propose Image2Sentence based Asymmetric zero-shot composed image retrieval (ISA), which takes advantage of the VL model and only relies on unlabeled images for composition learning.
  arXiv  Detail & Related papers  (2024-03-03T07:58:03Z)
• Symmetric Equilibrium Learning of VAEs [56.56929742714685]
  We view variational autoencoders (VAEs) as decoder-encoder pairs, which map distributions in the data space to distributions in the latent space and vice versa. We propose a Nash equilibrium learning approach, which is symmetric with respect to the encoder and decoder and allows learning VAEs in situations where both the data and the latent distributions are accessible only by sampling.
  arXiv  Detail & Related papers  (2023-07-19T10:27:34Z)
• Dictionary Learning under Symmetries via Group Representations [1.304892050913381]
  We study the problem of learning a dictionary that is invariant under a pre-specified group of transformations. We apply our paradigm to investigate the dictionary learning problem for the groups SO(2) and SO(3).
  arXiv  Detail & Related papers  (2023-05-31T04:54:06Z)
• Learning Invariant Subspaces of Koopman Operators--Part 2: Heterogeneous Dictionary Mixing to Approximate Subspace Invariance [0.0]
  This work builds on the models and concepts presented in part 1 to learn approximate dictionary representations of Koopman operators from data. We show that structured mixing of heterogeneous dictionary functions achieve the same accuracy and dimensional scaling as the deep-learning-based deepDMD algorithm.
  arXiv  Detail & Related papers  (2022-12-14T17:40:00Z)
• Learning Invariant Subspaces of Koopman Operators--Part 1: A Methodology for Demonstrating a Dictionary's Approximate Subspace Invariance [0.0]
  In a widely used algorithm, Extended Dynamic Mode Decomposition, the dictionary functions are drawn from a fixed class of functions. Deep learning combined with EDMD has been used to learn novel dictionary functions in an algorithm called deep dynamic mode decomposition (deepDMD) In this paper we analyze the learned dictionaries from deepDMD and explore the theoretical basis for their strong performance.
  arXiv  Detail & Related papers  (2022-12-14T17:33:52Z)
• Equivariance with Learned Canonicalization Functions [77.32483958400282]
  We show that learning a small neural network to perform canonicalization is better than using predefineds. Our experiments show that learning the canonicalization function is competitive with existing techniques for learning equivariant functions across many tasks.
  arXiv  Detail & Related papers  (2022-11-11T21:58:15Z)
• Statistical limits of dictionary learning: random matrix theory and the spectral replica method [28.54289139061295]
  We consider increasingly complex models of matrix denoising and dictionary learning in the Bayes-optimal setting. We introduce a novel combination of the replica method from statistical mechanics together with random matrix theory, coined spectral replica method.
  arXiv  Detail & Related papers  (2021-09-14T12:02:32Z)
• Learning Log-Determinant Divergences for Positive Definite Matrices [47.61701711840848]
  In this paper, we propose to learn similarity measures in a data-driven manner. We capitalize on the alphabeta-log-det divergence, which is a meta-divergence parametrized by scalars alpha and beta. Our key idea is to cast these parameters in a continuum and learn them from data.
  arXiv  Detail & Related papers  (2021-04-13T19:09:43Z)
• Anchor & Transform: Learning Sparse Embeddings for Large Vocabularies [60.285091454321055]
  We design a simple and efficient embedding algorithm that learns a small set of anchor embeddings and a sparse transformation matrix. On text classification, language modeling, and movie recommendation benchmarks, we show that ANT is particularly suitable for large vocabulary sizes.
  arXiv  Detail & Related papers  (2020-03-18T13:07:51Z)
• Ada-LISTA: Learned Solvers Adaptive to Varying Models [24.321416673430978]
  This work introduces an adaptive learned solver, termed Ada-LISTA, which receives pairs of signals and their corresponding dictionaries as inputs, and learns a universal architecture to serve them all. We prove that this scheme is guaranteed to solve sparse coding in linear rate for varying models, including dictionary perturbations and permutations. We also deploy Ada-LISTA to natural image inpainting, where the patch-masks vary spatially, thus requiring such an adaptation.
  arXiv  Detail & Related papers  (2020-01-23T11:34:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.