Deep Nonnegative Matrix Factorization with Beta Divergences

• URL: http://arxiv.org/abs/2309.08249v3
• Date: Mon, 18 Mar 2024 13:48:37 GMT
• Title: Deep Nonnegative Matrix Factorization with Beta Divergences
• Authors: Valentin Leplat, Le Thi Khanh Hien, Akwum Onwunta, Nicolas Gillis,
• Abstract summary: We develop new models and algorithms for deep NMF using some $beta$-divergences. We apply these techniques to the extraction of facial features, the identification of topics within document collections, and the identification of materials within hyperspectral images.
• Score: 14.639457874288412
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep Nonnegative Matrix Factorization (deep NMF) has recently emerged as a valuable technique for extracting multiple layers of features across different scales. However, all existing deep NMF models and algorithms have primarily centered their evaluation on the least squares error, which may not be the most appropriate metric for assessing the quality of approximations on diverse datasets. For instance, when dealing with data types such as audio signals and documents, it is widely acknowledged that $\beta$-divergences offer a more suitable alternative. In this paper, we develop new models and algorithms for deep NMF using some $\beta$-divergences, with a focus on the Kullback-Leibler divergence. Subsequently, we apply these techniques to the extraction of facial features, the identification of topics within document collections, and the identification of materials within hyperspectral images.

Related papers

• Targeted Visualization of the Backbone of Encoder LLMs [46.453758431767724]
  Attention based large language models (LLMs) are the state-of-the-art in natural language processing (NLP) Despite the success of encoder models, on which we focus in this work, they also bear several risks, including issues with bias or their susceptibility for adversarial attacks. We investigate the application of DeepView, a method for visualizing a part of the decision function together with a data set in two dimensions, to the NLP domain.
  arXiv  Detail & Related papers  (2024-03-26T12:51:02Z)
• Coseparable Nonnegative Tensor Factorization With T-CUR Decomposition [2.013220890731494]
  Nonnegative Matrix Factorization (NMF) is an important unsupervised learning method to extract meaningful features from data.<n>In this work, we provide an alternating selection method to select the coseparable core.<n>The results demonstrate the efficiency of coseparable NTF when compared to coseparable NMF.
  arXiv  Detail & Related papers  (2024-01-30T09:22:37Z)
• DiAD: A Diffusion-based Framework for Multi-class Anomaly Detection [55.48770333927732]
  We propose a Difusion-based Anomaly Detection (DiAD) framework for multi-class anomaly detection. It consists of a pixel-space autoencoder, a latent-space Semantic-Guided (SG) network with a connection to the stable diffusion's denoising network, and a feature-space pre-trained feature extractor. Experiments on MVTec-AD and VisA datasets demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2023-12-11T18:38:28Z)
• Contaminated Images Recovery by Implementing Non-negative Matrix Factorisation [0.0]
  We theoretically examine the robustness of the traditional NMF, HCNMF, and L2,1-NMF algorithms and execute sets of experiments to demonstrate the robustness on ORL and Extended YaleB datasets. Due to the computational cost of these approaches, our final models, such as the HCNMF and L2,1-NMF model, fail to converge within the parameters of this work.
  arXiv  Detail & Related papers  (2022-11-08T13:50:27Z)
• A consistent and flexible framework for deep matrix factorizations [17.49766938060264]
  We introduce two meaningful loss functions for deep MF and present a generic framework to solve the corresponding optimization problems. The models are successfully applied on both synthetic and real data, namely for hyperspectral unmixing and extraction of facial features.
  arXiv  Detail & Related papers  (2022-06-21T19:20:35Z)
• Inverting brain grey matter models with likelihood-free inference: a tool for trustable cytoarchitecture measurements [62.997667081978825]
  characterisation of the brain grey matter cytoarchitecture with quantitative sensitivity to soma density and volume remains an unsolved challenge in dMRI. We propose a new forward model, specifically a new system of equations, requiring a few relatively sparse b-shells. We then apply modern tools from Bayesian analysis known as likelihood-free inference (LFI) to invert our proposed model.
  arXiv  Detail & Related papers  (2021-11-15T09:08:27Z)
• Manifold Topology Divergence: a Framework for Comparing Data Manifolds [109.0784952256104]
  We develop a framework for comparing data manifold, aimed at the evaluation of deep generative models. Based on the Cross-Barcode, we introduce the Manifold Topology Divergence score (MTop-Divergence) We demonstrate that the MTop-Divergence accurately detects various degrees of mode-dropping, intra-mode collapse, mode invention, and image disturbance.
  arXiv  Detail & Related papers  (2021-06-08T00:30:43Z)
• M2TR: Multi-modal Multi-scale Transformers for Deepfake Detection [74.19291916812921]
  forged images generated by Deepfake techniques pose a serious threat to the trustworthiness of digital information. In this paper, we aim to capture the subtle manipulation artifacts at different scales for Deepfake detection. We introduce a high-quality Deepfake dataset, SR-DF, which consists of 4,000 DeepFake videos generated by state-of-the-art face swapping and facial reenactment methods.
  arXiv  Detail & Related papers  (2021-04-20T05:43:44Z)
• Feature Weighted Non-negative Matrix Factorization [92.45013716097753]
  We propose the Feature weighted Non-negative Matrix Factorization (FNMF) in this paper. FNMF learns the weights of features adaptively according to their importances. It can be solved efficiently with the suggested optimization algorithm.
  arXiv  Detail & Related papers  (2021-03-24T21:17:17Z)
• Brain Image Synthesis with Unsupervised Multivariate Canonical CSC$\ell_4$Net [122.8907826672382]
  We propose to learn dedicated features that cross both intre- and intra-modal variations using a novel CSC$ell_4$Net.
  arXiv  Detail & Related papers  (2021-03-22T05:19:40Z)
• Deep matrix factorizations [16.33338100088249]
  Deep matrix factorization (deep MF) was introduced to deal with the extraction of several layers of features. This paper presents the main models, algorithms, and applications of deep MF through a comprehensive literature review.
  arXiv  Detail & Related papers  (2020-10-01T13:19:01Z)
• Positive Semidefinite Matrix Factorization: A Connection with Phase Retrieval and Affine Rank Minimization [71.57324258813674]
  We show that PSDMF algorithms can be designed based on phase retrieval (PR) and affine rank minimization (ARM) algorithms. Motivated by this idea, we introduce a new family of PSDMF algorithms based on iterative hard thresholding (IHT)
  arXiv  Detail & Related papers  (2020-07-24T06:10:19Z)
• A Neural Network for Determination of Latent Dimensionality in Nonnegative Matrix Factorization [0.0]
  Non-negative Matrix Factorization (NMF) has proven to be a powerful unsupervised learning method for uncovering hidden features in complex and noisy data sets. We utilize a supervised machine learning approach in combination with a recent method for model determination, called NMFk, to determine the number of hidden features automatically.
  arXiv  Detail & Related papers  (2020-06-22T16:32:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.