Rank-one matrix estimation with groupwise heteroskedasticity

• URL: http://arxiv.org/abs/2106.11950v1
• Date: Tue, 22 Jun 2021 17:48:36 GMT
• Title: Rank-one matrix estimation with groupwise heteroskedasticity
• Authors: Joshua K. Behne and Galen Reeves
• Abstract summary: We study the problem of estimating a rank-one matrix from Gaussian observations where different blocks of the matrix are observed under different noise levels. We prove exact formulas for the minimum mean-squared error in estimating both the matrix and the latent variables. We derive an approximate message passing algorithm and a gradient descent algorithm and show empirically that these algorithms achieve the information-theoretic limits in certain regimes.
• Score: 5.202966939338455
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We study the problem of estimating a rank-one matrix from Gaussian observations where different blocks of the matrix are observed under different noise levels. This problem is motivated by applications in clustering and community detection where latent variables can be partitioned into a fixed number of known groups (e.g., users and items) and the blocks of the matrix correspond to different types of pairwise interactions (e.g., user-user, user-item, or item-item interactions). In the setting where the number of blocks is fixed while the number of variables tends to infinity, we prove asymptotically exact formulas for the minimum mean-squared error in estimating both the matrix and the latent variables. These formulas describe the weak recovery thresholds for the problem and reveal invariance properties with respect to certain scalings of the noise variance. We also derive an approximate message passing algorithm and a gradient descent algorithm and show empirically that these algorithms achieve the information-theoretic limits in certain regimes.

Related papers

• Empirical Bayes Linked Matrix Decomposition [0.0]
  We propose an empirical variational Bayesian approach to this problem. We describe an associated iterative imputation approach that is novel for the single-matrix context. We show that the method performs very well under different scenarios with respect to recovering underlying low-rank signal.
  arXiv  Detail & Related papers  (2024-08-01T02:13:11Z)
• Tensor cumulants for statistical inference on invariant distributions [49.80012009682584]
  We show that PCA becomes computationally hard at a critical value of the signal's magnitude. We define a new set of objects, which provide an explicit, near-orthogonal basis for invariants of a given degree. It also lets us analyze a new problem of distinguishing between different ensembles.
  arXiv  Detail & Related papers  (2024-04-29T14:33:24Z)
• Learning Graphical Factor Models with Riemannian Optimization [70.13748170371889]
  This paper proposes a flexible algorithmic framework for graph learning under low-rank structural constraints. The problem is expressed as penalized maximum likelihood estimation of an elliptical distribution. We leverage geometries of positive definite matrices and positive semi-definite matrices of fixed rank that are well suited to elliptical models.
  arXiv  Detail & Related papers  (2022-10-21T13:19:45Z)
• Comparative Study of Inference Methods for Interpolative Decomposition [4.913248451323163]
  We propose a probabilistic model with automatic relevance determination (ARD) for learning interpolative decomposition (ID) We evaluate the model on a variety of real-world datasets including CCLE $EC50$, CCLE $IC50$, Gene Body Methylation, and Promoter Methylation datasets with different sizes, and dimensions.
  arXiv  Detail & Related papers  (2022-06-29T11:37:05Z)
• Lattice-Based Methods Surpass Sum-of-Squares in Clustering [98.46302040220395]
  Clustering is a fundamental primitive in unsupervised learning. Recent work has established lower bounds against the class of low-degree methods. We show that, perhaps surprisingly, this particular clustering model textitdoes not exhibit a statistical-to-computational gap.
  arXiv  Detail & Related papers  (2021-12-07T18:50:17Z)
• Learning Noise Transition Matrix from Only Noisy Labels via Total Variation Regularization [88.91872713134342]
  We propose a theoretically grounded method that can estimate the noise transition matrix and learn a classifier simultaneously. We show the effectiveness of the proposed method through experiments on benchmark and real-world datasets.
  arXiv  Detail & Related papers  (2021-02-04T05:09:18Z)
• All-or-nothing statistical and computational phase transitions in sparse spiked matrix estimation [35.035853993422506]
  We analyze the approximate message passing algorithm in a sparse regime. We find all-or-nothing phase transitions for the minimum and mean-square errors. In the sparse regime the statistical-to-algorithmic gap diverges indicating that sparse recovery is hard for approximate message passing.
  arXiv  Detail & Related papers  (2020-06-14T18:38:34Z)
• Instability, Computational Efficiency and Statistical Accuracy [101.32305022521024]
  We develop a framework that yields statistical accuracy based on interplay between the deterministic convergence rate of the algorithm at the population level, and its degree of (instability) when applied to an empirical object based on $n$ samples. We provide applications of our general results to several concrete classes of models, including Gaussian mixture estimation, non-linear regression models, and informative non-response models.
  arXiv  Detail & Related papers  (2020-05-22T22:30:52Z)
• Information-Theoretic Limits for the Matrix Tensor Product [8.206394018475708]
  This paper studies a high-dimensional inference problem involving the matrix tensor product of random matrices. On the technical side, this paper introduces some new techniques for the analysis of high-dimensional matrix-preserving signals.
  arXiv  Detail & Related papers  (2020-05-22T17:03:48Z)
• Asymptotic Analysis of an Ensemble of Randomly Projected Linear Discriminants [94.46276668068327]
  In [1], an ensemble of randomly projected linear discriminants is used to classify datasets. We develop a consistent estimator of the misclassification probability as an alternative to the computationally-costly cross-validation estimator. We also demonstrate the use of our estimator for tuning the projection dimension on both real and synthetic data.
  arXiv  Detail & Related papers  (2020-04-17T12:47:04Z)
• A Block Coordinate Descent-based Projected Gradient Algorithm for Orthogonal Non-negative Matrix Factorization [0.0]
  This article utilizes the projected gradient method (PG) for a non-negative matrix factorization problem (NMF) We penalise the orthonormality constraints and apply the PG method via a block coordinate descent approach.
  arXiv  Detail & Related papers  (2020-03-23T13:24:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.