A majorization-minimization algorithm for nonnegative binary matrix
factorization
- URL: http://arxiv.org/abs/2204.09741v1
- Date: Wed, 20 Apr 2022 18:53:34 GMT
- Title: A majorization-minimization algorithm for nonnegative binary matrix
factorization
- Authors: Paul Magron, C\'edric F\'evotte
- Abstract summary: This paper tackles the problem of decomposing binary data using matrix factorization.
We factorize the Bernoulli parameter and consider an additional Beta prior on one of the factors to further improve the model's expressive power.
Experiments conducted on three public binary datasets show that our approach offers an excellent trade-off between prediction performance, computational complexity, and interpretability.
- Score: 9.457368716414077
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: This paper tackles the problem of decomposing binary data using matrix
factorization. We consider the family of mean-parametrized Bernoulli models, a
class of generative models that are well suited for modeling binary data and
enables interpretability of the factors. We factorize the Bernoulli parameter
and consider an additional Beta prior on one of the factors to further improve
the model's expressive power. While similar models have been proposed in the
literature, they only exploit the Beta prior as a proxy to ensure a valid
Bernoulli parameter in a Bayesian setting; in practice it reduces to a uniform
or uninformative prior. Besides, estimation in these models has focused on
costly Bayesian inference. In this paper, we propose a simple yet very
efficient majorization-minimization algorithm for maximum a posteriori
estimation. Our approach leverages the Beta prior whose parameters can be tuned
to improve performance in matrix completion tasks. Experiments conducted on
three public binary datasets show that our approach offers an excellent
trade-off between prediction performance, computational complexity, and
interpretability.
Related papers
- Banded Square Root Matrix Factorization for Differentially Private Model Training [3.6371628922281305]
We present a new matrix factorization approach, BSR, which overcomes this computational bottleneck.
By exploiting properties of the standard matrix square root, BSR allows to efficiently handle also large-scale problems.
Our numerical experiments demonstrate that models trained using BSR perform on par with the best existing methods, while completely avoiding their computational overhead.
arXiv Detail & Related papers (2024-05-22T15:47:35Z) - Quantization of Large Language Models with an Overdetermined Basis [73.79368761182998]
We introduce an algorithm for data quantization based on the principles of Kashin representation.
Our findings demonstrate that Kashin Quantization achieves competitive or superior quality in model performance.
arXiv Detail & Related papers (2024-04-15T12:38:46Z) - Efficient and Generalizable Certified Unlearning: A Hessian-free Recollection Approach [8.875278412741695]
Machine unlearning strives to uphold the data owners' right to be forgotten by enabling models to selectively forget specific data.
We develop an algorithm that achieves near-instantaneous unlearning as it only requires a vector addition operation.
arXiv Detail & Related papers (2024-04-02T07:54:18Z) - Poisson Process for Bayesian Optimization [126.51200593377739]
We propose a ranking-based surrogate model based on the Poisson process and introduce an efficient BO framework, namely Poisson Process Bayesian Optimization (PoPBO)
Compared to the classic GP-BO method, our PoPBO has lower costs and better robustness to noise, which is verified by abundant experiments.
arXiv Detail & Related papers (2024-02-05T02:54:50Z) - Bayesian Beta-Bernoulli Process Sparse Coding with Deep Neural Networks [11.937283219047984]
Several approximate inference methods have been proposed for deep discrete latent variable models.
We propose a non-parametric iterative algorithm for learning discrete latent representations in such deep models.
We evaluate our method across datasets with varying characteristics and compare our results to current amortized approximate inference methods.
arXiv Detail & Related papers (2023-03-14T20:50:12Z) - Sparse high-dimensional linear regression with a partitioned empirical
Bayes ECM algorithm [62.997667081978825]
We propose a computationally efficient and powerful Bayesian approach for sparse high-dimensional linear regression.
Minimal prior assumptions on the parameters are used through the use of plug-in empirical Bayes estimates.
The proposed approach is implemented in the R package probe.
arXiv Detail & Related papers (2022-09-16T19:15:50Z) - Quasi Black-Box Variational Inference with Natural Gradients for
Bayesian Learning [84.90242084523565]
We develop an optimization algorithm suitable for Bayesian learning in complex models.
Our approach relies on natural gradient updates within a general black-box framework for efficient training with limited model-specific derivations.
arXiv Detail & Related papers (2022-05-23T18:54:27Z) - Scalable Combinatorial Bayesian Optimization with Tractable Statistical
models [44.25245545568633]
We study the problem of optimizing blackbox functions over Relaxation spaces (e.g., sets, sequences, trees, and graphs)
Based on recent advances in submodular relaxation, we study an approach as Parametrized Submodular (PSR) towards the goal of improving the scalability and accuracy of solving AFO problems for BOCS model.
Experiments on diverse benchmark problems show significant improvements with PSR for BOCS model.
arXiv Detail & Related papers (2020-08-18T22:56:46Z) - Slice Sampling for General Completely Random Measures [74.24975039689893]
We present a novel Markov chain Monte Carlo algorithm for posterior inference that adaptively sets the truncation level using auxiliary slice variables.
The efficacy of the proposed algorithm is evaluated on several popular nonparametric models.
arXiv Detail & Related papers (2020-06-24T17:53:53Z) - Efficient Ensemble Model Generation for Uncertainty Estimation with
Bayesian Approximation in Segmentation [74.06904875527556]
We propose a generic and efficient segmentation framework to construct ensemble segmentation models.
In the proposed method, ensemble models can be efficiently generated by using the layer selection method.
We also devise a new pixel-wise uncertainty loss, which improves the predictive performance.
arXiv Detail & Related papers (2020-05-21T16:08:38Z) - Gaussian Process Boosting [6.85316573653194]
We introduce a novel way to combine boosting with Gaussian process and mixed effects models.
We obtain increased prediction accuracy compared to existing approaches on simulated and real-world data sets.
arXiv Detail & Related papers (2020-04-06T13:19:54Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.