Algorithms for Nonnegative Matrix Factorization with the Kullback-Leibler Divergence

• URL: http://arxiv.org/abs/2010.01935v2
• Date: Sat, 17 Apr 2021 07:27:02 GMT
• Title: Algorithms for Nonnegative Matrix Factorization with the Kullback-Leibler Divergence
• Authors: Le Thi Khanh Hien, Nicolas Gillis
• Abstract summary: Kullback-Leibler (KL) divergence is one of the most widely used objective function for nonnegative matrix factorization (NMF) We propose three new algorithms that guarantee the non-increasingness of the objective function. We conduct extensive numerical experiments to provide a comprehensive picture of the performances of the KL NMF algorithms.
• Score: 20.671178429005973
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Nonnegative matrix factorization (NMF) is a standard linear dimensionality reduction technique for nonnegative data sets. In order to measure the discrepancy between the input data and the low-rank approximation, the Kullback-Leibler (KL) divergence is one of the most widely used objective function for NMF. It corresponds to the maximum likehood estimator when the underlying statistics of the observed data sample follows a Poisson distribution, and KL NMF is particularly meaningful for count data sets, such as documents or images. In this paper, we first collect important properties of the KL objective function that are essential to study the convergence of KL NMF algorithms. Second, together with reviewing existing algorithms for solving KL NMF, we propose three new algorithms that guarantee the non-increasingness of the objective function. We also provide a global convergence guarantee for one of our proposed algorithms. Finally, we conduct extensive numerical experiments to provide a comprehensive picture of the performances of the KL NMF algorithms.

Related papers

• Exponentially Convergent Algorithms for Supervised Matrix Factorization [2.1485350418225244]
  Supervised factorization (SMF) is a machine learning method that converges extraction and classification tasks. Our paper provides a novel framework that 'lifts' SMF as a low-rank estimation problem in a combined factor space estimation.
  arXiv  Detail & Related papers  (2023-11-18T23:24:02Z)
• Stratified-NMF for Heterogeneous Data [8.174199227297514]
  We propose a modified NMF objective, Stratified-NMF, that simultaneously learns strata-dependent statistics and a shared topics matrix. We apply our method to three real world datasets and empirically investigate their learned features.
  arXiv  Detail & Related papers  (2023-11-17T00:34:41Z)
• Contaminated Images Recovery by Implementing Non-negative Matrix Factorisation [0.0]
  We theoretically examine the robustness of the traditional NMF, HCNMF, and L2,1-NMF algorithms and execute sets of experiments to demonstrate the robustness on ORL and Extended YaleB datasets. Due to the computational cost of these approaches, our final models, such as the HCNMF and L2,1-NMF model, fail to converge within the parameters of this work.
  arXiv  Detail & Related papers  (2022-11-08T13:50:27Z)
• Supervised Class-pairwise NMF for Data Representation and Classification [2.7320863258816512]
  Non-negative Matrix factorization (NMF) based methods add new terms to the cost function to adapt the model to specific tasks. NMF method adopts unsupervised approaches to estimate the factorizing matrices.
  arXiv  Detail & Related papers  (2022-09-28T04:33:03Z)
• Log-based Sparse Nonnegative Matrix Factorization for Data Representation [55.72494900138061]
  Nonnegative matrix factorization (NMF) has been widely studied in recent years due to its effectiveness in representing nonnegative data with parts-based representations. We propose a new NMF method with log-norm imposed on the factor matrices to enhance the sparseness. A novel column-wisely sparse norm, named $ell_2,log$-(pseudo) norm, is proposed to enhance the robustness of the proposed method.
  arXiv  Detail & Related papers  (2022-04-22T11:38:10Z)
• Feature Weighted Non-negative Matrix Factorization [92.45013716097753]
  We propose the Feature weighted Non-negative Matrix Factorization (FNMF) in this paper. FNMF learns the weights of features adaptively according to their importances. It can be solved efficiently with the suggested optimization algorithm.
  arXiv  Detail & Related papers  (2021-03-24T21:17:17Z)
• Entropy Minimizing Matrix Factorization [102.26446204624885]
  Nonnegative Matrix Factorization (NMF) is a widely-used data analysis technique, and has yielded impressive results in many real-world tasks. In this study, an Entropy Minimizing Matrix Factorization framework (EMMF) is developed to tackle the above problem. Considering that the outliers are usually much less than the normal samples, a new entropy loss function is established for matrix factorization.
  arXiv  Detail & Related papers  (2021-03-24T21:08:43Z)
• Self-supervised Symmetric Nonnegative Matrix Factorization [82.59905231819685]
  Symmetric nonnegative factor matrix (SNMF) has demonstrated to be a powerful method for data clustering. Inspired by ensemble clustering that aims to seek better clustering results, we propose self-supervised SNMF (S$3$NMF) We take advantage of the sensitivity to code characteristic of SNMF, without relying on any additional information.
  arXiv  Detail & Related papers  (2021-03-02T12:47:40Z)
• Positive Semidefinite Matrix Factorization: A Connection with Phase Retrieval and Affine Rank Minimization [71.57324258813674]
  We show that PSDMF algorithms can be designed based on phase retrieval (PR) and affine rank minimization (ARM) algorithms. Motivated by this idea, we introduce a new family of PSDMF algorithms based on iterative hard thresholding (IHT)
  arXiv  Detail & Related papers  (2020-07-24T06:10:19Z)
• Model Fusion with Kullback--Leibler Divergence [58.20269014662046]
  We propose a method to fuse posterior distributions learned from heterogeneous datasets. Our algorithm relies on a mean field assumption for both the fused model and the individual dataset posteriors.
  arXiv  Detail & Related papers  (2020-07-13T03:27:45Z)
• A Neural Network for Determination of Latent Dimensionality in Nonnegative Matrix Factorization [0.0]
  Non-negative Matrix Factorization (NMF) has proven to be a powerful unsupervised learning method for uncovering hidden features in complex and noisy data sets. We utilize a supervised machine learning approach in combination with a recent method for model determination, called NMFk, to determine the number of hidden features automatically.
  arXiv  Detail & Related papers  (2020-06-22T16:32:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.