Atom dimension adaptation for infinite set dictionary learning

• URL: http://arxiv.org/abs/2409.06831v1
• Date: Tue, 10 Sep 2024 19:18:03 GMT
• Title: Atom dimension adaptation for infinite set dictionary learning
• Abstract summary: We propose a method for adaptively adjusting the size of set-atoms in dictionary learning. The proposed algorithm not only decreases the representation error, but also improves anomaly detection.
• Score: 0.6554326244334866
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent work on dictionary learning with set-atoms has shown benefits in anomaly detection. Instead of viewing an atom as a single vector, these methods allow building sparse representations with atoms taken from a set around a central vector; the set can be a cone or may have a probability distribution associated to it. We propose a method for adaptively adjusting the size of set-atoms in Gaussian and cone dictionary learning. The purpose of the algorithm is to match the atom sizes with their contribution in representing the signals. The proposed algorithm not only decreases the representation error, but also improves anomaly detection, for a class of anomalies called `dependency'. We obtain better detection performance than state-of-the-art methods.

Related papers

• Sparse Fine-Tuning of Transformers for Generative Tasks [25.83571722912947]
  We introduce a fine-tuning framework inspired by sparse coding, where fine-tuned features are represented as a sparse combination of basic elements.<n>The feature dictionary atoms function as fundamental building blocks of the representation, and tuning atoms allows for seamless adaptation to downstream tasks.
  arXiv  Detail & Related papers  (2025-07-14T23:03:24Z)
• Categorical Data Clustering via Value Order Estimated Distance Metric Learning [53.28598689867732]
  This paper introduces a novel order distance metric learning approach to intuitively represent categorical attribute values.<n>A new joint learning paradigm is developed to alternatively perform clustering and order distance metric learning.<n>The proposed method achieves superior clustering accuracy on categorical and mixed datasets.
  arXiv  Detail & Related papers  (2024-11-19T08:23:25Z)
• Convergence of alternating minimisation algorithms for dictionary learning [4.5687771576879594]
  We derive sufficient conditions for the convergence of two popular alternating minimisation algorithms for dictionary learning. We show that given a well-behaved initialisation that is either within distance at most $1/log(K)$ to the generating dictionary or has a special structure ensuring that each element of the initialisation only points to one generating element, both algorithms will converge with geometric convergence rate to the generating dictionary.
  arXiv  Detail & Related papers  (2023-04-04T12:58:47Z)
• Efficient algorithms to solve atom reconfiguration problems. II. The assignment-rerouting-ordering (aro) algorithm [51.02512563152503]
  atom reconfiguration problems require solving an atom problem quickly and efficiently. A typical approach to solve atom reconfiguration problems is to use an assignment algorithm to determine which atoms to move to which traps. This approach does not optimize for the number of displaced atoms nor the number of times each atom is displaced. We propose the assignment-rerouting-ordering (aro) algorithm to improve the performance of assignment-based algorithms in solving atom reconfiguration problems.
  arXiv  Detail & Related papers  (2022-12-11T19:48:25Z)
• Efficient algorithms to solve atom reconfiguration problems. I. The redistribution-reconfiguration (red-rec) algorithm [51.02512563152503]
  We numerically quantify the performance of the red-rec algorithm, both in the absence and in the presence of loss. We show that the number of traps required to prepare a compact-centered configuration of atoms on a grid with a mean success probability of one half scales as the 3/2 power of the number of desired atoms. The red-rec algorithm admits an efficient implementation that can readily be deployed on real-time control systems.
  arXiv  Detail & Related papers  (2022-12-07T19:00:01Z)
• Simple Alternating Minimization Provably Solves Complete Dictionary Learning [13.056764072568749]
  This paper focuses on complete dictionary problem, where the goal is to reparametrize a set of given signals as linear combinations of atoms from a learned dictionary. There are two main challenges faced by theoretical and practical dictionary learning: the lack of theoretical guarantees for practically-used algorithms, and poor scalability when dealing with huge-scale datasets.
  arXiv  Detail & Related papers  (2022-10-23T18:30:45Z)
• Object Representations as Fixed Points: Training Iterative Refinement Algorithms with Implicit Differentiation [88.14365009076907]
  Iterative refinement is a useful paradigm for representation learning. We develop an implicit differentiation approach that improves the stability and tractability of training.
  arXiv  Detail & Related papers  (2022-07-02T10:00:35Z)
• Self-Supervised Training with Autoencoders for Visual Anomaly Detection [61.62861063776813]
  We focus on a specific use case in anomaly detection where the distribution of normal samples is supported by a lower-dimensional manifold. We adapt a self-supervised learning regime that exploits discriminative information during training but focuses on the submanifold of normal examples. We achieve a new state-of-the-art result on the MVTec AD dataset -- a challenging benchmark for visual anomaly detection in the manufacturing domain.
  arXiv  Detail & Related papers  (2022-06-23T14:16:30Z)
• Bayesian Graph Contrastive Learning [55.36652660268726]
  We propose a novel perspective of graph contrastive learning methods showing random augmentations leads to encoders. Our proposed method represents each node by a distribution in the latent space in contrast to existing techniques which embed each node to a deterministic vector. We show a considerable improvement in performance compared to existing state-of-the-art methods on several benchmark datasets.
  arXiv  Detail & Related papers  (2021-12-15T01:45:32Z)
• Dictionary Learning Using Rank-One Atomic Decomposition (ROAD) [6.367823813868024]
  Dictionary learning aims at seeking a dictionary under which the training data can be sparsely represented. Road outperforms other benchmark algorithms for both synthetic data and real data.
  arXiv  Detail & Related papers  (2021-10-25T10:29:52Z)
• Sparse Quadratic Optimisation over the Stiefel Manifold with Application to Permutation Synchronisation [71.27989298860481]
  We address the non- optimisation problem of finding a matrix on the Stiefel manifold that maximises a quadratic objective function. We propose a simple yet effective sparsity-promoting algorithm for finding the dominant eigenspace matrix.
  arXiv  Detail & Related papers  (2021-09-30T19:17:35Z)
• Kernel Density Estimation by Stagewise Algorithm with a Simple Dictionary [0.0]
  This paper studies kernel density estimation by stagewise algorithm with a simple dictionary on U-divergence. We randomly split an i.i.d. sample into two disjoint sets, one for constructing the kernels in the dictionary and the other for evaluating the estimator.
  arXiv  Detail & Related papers  (2021-07-27T17:05:06Z)
• Sparse PCA via $l_{2,p}$-Norm Regularization for Unsupervised Feature Selection [138.97647716793333]
  We propose a simple and efficient unsupervised feature selection method, by combining reconstruction error with $l_2,p$-norm regularization. We present an efficient optimization algorithm to solve the proposed unsupervised model, and analyse the convergence and computational complexity of the algorithm theoretically.
  arXiv  Detail & Related papers  (2020-12-29T04:08:38Z)
• Anchor & Transform: Learning Sparse Embeddings for Large Vocabularies [60.285091454321055]
  We design a simple and efficient embedding algorithm that learns a small set of anchor embeddings and a sparse transformation matrix. On text classification, language modeling, and movie recommendation benchmarks, we show that ANT is particularly suitable for large vocabulary sizes.
  arXiv  Detail & Related papers  (2020-03-18T13:07:51Z)
• Sparse Optimization on General Atomic Sets: Greedy and Forward-Backward Algorithms [0.0]
  We show that greedy algorithms can attain strong approximation guarantees while maintaining sparsity. In particular, we show that in many settings of interest the greedy algorithm can attain strong approximation guarantees while maintaining sparsity. Secondly, we define an alternate notion of weak submodularity, which we show is tightly related to the more familiar version that has been used to prove earlier linear convergence rates.
  arXiv  Detail & Related papers  (2019-12-26T20:52:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.