Representation Learning via Cauchy Convolutional Sparse Coding

• URL: http://arxiv.org/abs/2008.03473v1
• Date: Sat, 8 Aug 2020 08:21:44 GMT
• Title: Representation Learning via Cauchy Convolutional Sparse Coding
• Authors: Perla Mayo, Oktay Karaku\c{s}, Robin Holmes and Alin Achim
• Abstract summary: Convolutional Sparse Coding (CSC) enables unsupervised learning of features by jointly optimising both an (ell)-norm fidelity term and a sparsity enforcing penalty. This work investigates using a regularisation term derived from an assumed Cauchy prior for the coefficients of the feature maps of a CSC generative model. The sparsity penalty term resulting from this prior is solved via its proximal operator, which is then applied iteratively, element-wise, on the coefficients of the feature maps to optimise the CSC cost function.
• Score: 7.784354802855044
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In representation learning, Convolutional Sparse Coding (CSC) enables unsupervised learning of features by jointly optimising both an \(\ell_2\)-norm fidelity term and a sparsity enforcing penalty. This work investigates using a regularisation term derived from an assumed Cauchy prior for the coefficients of the feature maps of a CSC generative model. The sparsity penalty term resulting from this prior is solved via its proximal operator, which is then applied iteratively, element-wise, on the coefficients of the feature maps to optimise the CSC cost function. The performance of the proposed Iterative Cauchy Thresholding (ICT) algorithm in reconstructing natural images is compared against the common choice of \(\ell_1\)-norm optimised via soft and hard thresholding. ICT outperforms IHT and IST in most of these reconstruction experiments across various datasets, with an average PSNR of up to 11.30 and 7.04 above ISTA and IHT respectively.

Related papers

• Stochastic Primal-Dual Double Block-Coordinate for Two-way Partial AUC Maximization [56.805574957824135]
  Two-way partial AUCAUC is a critical performance metric for binary classification with imbalanced data.<n>Existing algorithms for TPAUC optimization remain under-explored.<n>We introduce two innovative double-coordinate block-coordinate algorithms for TPAUC optimization.
  arXiv  Detail & Related papers  (2025-05-28T03:55:05Z)
• Adaptive function approximation based on the Discrete Cosine Transform (DCT) [2.2713084727838115]
  This paper studies a supervised learning to obtain the approximation coefficients, instead of using the Discrete Cosine Transform (DCT) Due to the finite dynamics and gradientity of the cosine basis functions, simple algorithms, such as the Normalized Least Mean Squares (NLMS) can benefit from it. This paper celebrates the 50th anniversary of the publication of the DCT by Nasir Ahmed in 1973.
  arXiv  Detail & Related papers  (2023-09-01T15:31:26Z)
• Decoupled Training for Long-Tailed Classification With Stochastic Representations [15.990318581975435]
  Decoupling representation learning and learning has been shown to be effective in classification with long-tailed data. We first apply Weight Averaging (SWA), an optimization technique for improving generalization of deep neural networks, to obtain better generalizing feature extractors for long-tailed classification. We then propose a novel classifier re-training algorithm based on perturbed representation obtained from the SWA-Gaussian, a Gaussian SWA, and a self-distillation strategy.
  arXiv  Detail & Related papers  (2023-04-19T05:35:09Z)
• Improved Algorithms for Neural Active Learning [74.89097665112621]
  We improve the theoretical and empirical performance of neural-network(NN)-based active learning algorithms for the non-parametric streaming setting. We introduce two regret metrics by minimizing the population loss that are more suitable in active learning than the one used in state-of-the-art (SOTA) related work.
  arXiv  Detail & Related papers  (2022-10-02T05:03:38Z)
• Reinforcement Learning from Partial Observation: Linear Function Approximation with Provable Sample Efficiency [111.83670279016599]
  We study reinforcement learning for partially observed decision processes (POMDPs) with infinite observation and state spaces. We make the first attempt at partial observability and function approximation for a class of POMDPs with a linear structure.
  arXiv  Detail & Related papers  (2022-04-20T21:15:38Z)
• Contrastive Conditional Neural Processes [45.70735205041254]
  Conditional Neural Processes(CNPs) bridge neural networks with probabilistic inference to approximate functions of Processes under meta-learning settings. Two auxiliary contrastive branches are set up hierarchically, namely in-instantiation temporal contrastive learning(tt TCL) and cross-instantiation function contrastive learning(tt FCL) We empirically show that tt TCL captures high-level abstraction of observations, whereas tt FCL helps identify underlying functions, which in turn provides more efficient representations.
  arXiv  Detail & Related papers  (2022-03-08T10:08:45Z)
• Adversarial Feature Augmentation and Normalization for Visual Recognition [109.6834687220478]
  Recent advances in computer vision take advantage of adversarial data augmentation to ameliorate the generalization ability of classification models. Here, we present an effective and efficient alternative that advocates adversarial augmentation on intermediate feature embeddings. We validate the proposed approach across diverse visual recognition tasks with representative backbone networks.
  arXiv  Detail & Related papers  (2021-03-22T20:36:34Z)
• Out-of-distribution Generalization via Partial Feature Decorrelation [72.96261704851683]
  We present a novel Partial Feature Decorrelation Learning (PFDL) algorithm, which jointly optimize a feature decomposition network and the target image classification model. The experiments on real-world datasets demonstrate that our method can improve the backbone model's accuracy on OOD image classification datasets.
  arXiv  Detail & Related papers  (2020-07-30T05:48:48Z)
• Fibonacci and k-Subsecting Recursive Feature Elimination [2.741266294612776]
  Feature selection is a data mining task with the potential of speeding up classification algorithms. We propose two novel algorithms called Fibonacci- and k-Subsecting Recursive Feature Elimination. Results show that Fibonacci and k-Subsecting Recursive Feature Elimination are capable of selecting a smaller subset of features much faster than standard RFE.
  arXiv  Detail & Related papers  (2020-07-29T15:53:04Z)
• New advances in enumerative biclustering algorithms with online partitioning [80.22629846165306]
  This paper further extends RIn-Close_CVC, a biclustering algorithm capable of performing an efficient, complete, correct and non-redundant enumeration of maximal biclusters with constant values on columns in numerical datasets. The improved algorithm is called RIn-Close_CVC3, keeps those attractive properties of RIn-Close_CVC, and is characterized by: a drastic reduction in memory usage; a consistent gain in runtime.
  arXiv  Detail & Related papers  (2020-03-07T14:54:26Z)
• Supervised Learning for Non-Sequential Data: A Canonical Polyadic Decomposition Approach [85.12934750565971]
  Efficient modelling of feature interactions underpins supervised learning for non-sequential tasks. To alleviate this issue, it has been proposed to implicitly represent the model parameters as a tensor. For enhanced expressiveness, we generalize the framework to allow feature mapping to arbitrarily high-dimensional feature vectors.
  arXiv  Detail & Related papers  (2020-01-27T22:38:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.