Low-rank Bayesian matrix completion via geodesic Hamiltonian Monte Carlo on Stiefel manifolds

• URL: http://arxiv.org/abs/2410.20318v1
• Date: Sun, 27 Oct 2024 03:12:53 GMT
• Title: Low-rank Bayesian matrix completion via geodesic Hamiltonian Monte Carlo on Stiefel manifolds
• Authors: Tiangang Cui, Alex Gorodetsky,
• Abstract summary: We present a new sampling-based approach for enabling efficient computation of low-rank Bayesian matrix completion. We show that our approach resolves the sampling difficulties encountered by standard Gibbs samplers for the common two-matrix factorization used in matrix completion. Numerical examples demonstrate superior sampling performance, including better mixing and faster convergence to a stationary distribution.
• Score: 0.18416014644193066
• License:
• Abstract: We present a new sampling-based approach for enabling efficient computation of low-rank Bayesian matrix completion and quantifying the associated uncertainty. Firstly, we design a new prior model based on the singular-value-decomposition (SVD) parametrization of low-rank matrices. Our prior is analogous to the seminal nuclear-norm regularization used in non-Bayesian setting and enforces orthogonality in the factor matrices by constraining them to Stiefel manifolds. Then, we design a geodesic Hamiltonian Monte Carlo (-within-Gibbs) algorithm for generating posterior samples of the SVD factor matrices. We demonstrate that our approach resolves the sampling difficulties encountered by standard Gibbs samplers for the common two-matrix factorization used in matrix completion. More importantly, the geodesic Hamiltonian sampler allows for sampling in cases with more general likelihoods than the typical Gaussian likelihood and Gaussian prior assumptions adopted in most of the existing Bayesian matrix completion literature. We demonstrate an applications of our approach to fit the categorical data of a mice protein dataset and the MovieLens recommendation problem. Numerical examples demonstrate superior sampling performance, including better mixing and faster convergence to a stationary distribution. Moreover, they demonstrate improved accuracy on the two real-world benchmark problems we considered.

Related papers

• Concentration of a sparse Bayesian model with Horseshoe prior in estimating high-dimensional precision matrix [0.0]
  We provide theoretical results for the tempered posterior with the fully specified horseshoe prior in high-dimensional settings. We also provide novel theoretical results for model misspecification, offering a general oracle inequality for the posterior.
  arXiv  Detail & Related papers  (2024-06-20T12:50:54Z)
• Symmetric Matrix Completion with ReLU Sampling [15.095194065320987]
  We study the problem of symmetric positive semi-definite low-rank matrix completion (MC) with entry-dependent sampling. In particular, we consider rectified linear unit (ReLU) sampling, where only stationary points are observed.
  arXiv  Detail & Related papers  (2024-06-09T15:14:53Z)
• Probabilistic Unrolling: Scalable, Inverse-Free Maximum Likelihood Estimation for Latent Gaussian Models [69.22568644711113]
  We introduce probabilistic unrolling, a method that combines Monte Carlo sampling with iterative linear solvers to circumvent matrix inversions. Our theoretical analyses reveal that unrolling and backpropagation through the iterations of the solver can accelerate gradient estimation for maximum likelihood estimation. In experiments on simulated and real data, we demonstrate that probabilistic unrolling learns latent Gaussian models up to an order of magnitude faster than gradient EM, with minimal losses in model performance.
  arXiv  Detail & Related papers  (2023-06-05T21:08:34Z)
• Manifold Gaussian Variational Bayes on the Precision Matrix [70.44024861252554]
  We propose an optimization algorithm for Variational Inference (VI) in complex models. We develop an efficient algorithm for Gaussian Variational Inference whose updates satisfy the positive definite constraint on the variational covariance matrix. Due to its black-box nature, MGVBP stands as a ready-to-use solution for VI in complex models.
  arXiv  Detail & Related papers  (2022-10-26T10:12:31Z)
• Langevin Monte Carlo for Contextual Bandits [72.00524614312002]
  Langevin Monte Carlo Thompson Sampling (LMC-TS) is proposed to directly sample from the posterior distribution in contextual bandits. We prove that the proposed algorithm achieves the same sublinear regret bound as the best Thompson sampling algorithms for a special case of contextual bandits.
  arXiv  Detail & Related papers  (2022-06-22T17:58:23Z)
• Semi-Supervised Subspace Clustering via Tensor Low-Rank Representation [64.49871502193477]
  We propose a novel semi-supervised subspace clustering method, which is able to simultaneously augment the initial supervisory information and construct a discriminative affinity matrix. Comprehensive experimental results on six commonly-used benchmark datasets demonstrate the superiority of our method over state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-05-21T01:47:17Z)
• Spectral clustering under degree heterogeneity: a case for the random walk Laplacian [83.79286663107845]
  This paper shows that graph spectral embedding using the random walk Laplacian produces vector representations which are completely corrected for node degree. In the special case of a degree-corrected block model, the embedding concentrates about K distinct points, representing communities.
  arXiv  Detail & Related papers  (2021-05-03T16:36:27Z)
• Multilevel Gibbs Sampling for Bayesian Regression [6.2997667081978825]
  The level hierarchy of data matrices is created by clustering the features and/or samples of data matrices. The use of correlated samples is investigated for variance reduction to improve the convergence of the Markov Chain. Speed-up is achieved for almost all of them without significant loss in predictive performance.
  arXiv  Detail & Related papers  (2020-09-25T11:18:17Z)
• Bayesian learning of orthogonal embeddings for multi-fidelity Gaussian Processes [3.564709604457361]
  "Projection" mapping consists of an orthonormal matrix that is considered a priori unknown and needs to be inferred jointly with the GP parameters. We extend the proposed framework to multi-fidelity models using GPs including the scenarios of training multiple outputs together. The benefits of our proposed framework, are illustrated on the computationally challenging three-dimensional aerodynamic optimization of a last-stage blade for an industrial gas turbine.
  arXiv  Detail & Related papers  (2020-08-05T22:28:53Z)
• Efficient MCMC Sampling for Bayesian Matrix Factorization by Breaking Posterior Symmetries [1.3858051019755282]
  We propose a simple modification to the prior choice that provably breaks these symmetries and maintains/improves accuracy. We show that using non-zero linearly independent prior means significantly lowers the autocorrelation of MCMC samples, and can also lead to lower reconstruction errors.
  arXiv  Detail & Related papers  (2020-06-08T00:25:48Z)
• Optimal Iterative Sketching with the Subsampled Randomized Hadamard Transform [64.90148466525754]
  We study the performance of iterative sketching for least-squares problems. We show that the convergence rate for Haar and randomized Hadamard matrices are identical, andally improve upon random projections. These techniques may be applied to other algorithms that employ randomized dimension reduction.
  arXiv  Detail & Related papers  (2020-02-03T16:17:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.