Learnable Graph-regularization for Matrix Decomposition

• URL: http://arxiv.org/abs/2010.08513v1
• Date: Fri, 16 Oct 2020 17:12:39 GMT
• Title: Learnable Graph-regularization for Matrix Decomposition
• Authors: Penglong Zhai and Shihua Zhang
• Abstract summary: We propose a learnable graph-regularization model for matrix decomposition. It builds a bridge between graph-regularized methods and probabilistic matrix decomposition models. It learns two graphical structures in real-time in an iterative manner via sparse precision matrix estimation.
• Score: 5.9394103049943485
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Low-rank approximation models of data matrices have become important machine learning and data mining tools in many fields including computer vision, text mining, bioinformatics and many others. They allow for embedding high-dimensional data into low-dimensional spaces, which mitigates the effects of noise and uncovers latent relations. In order to make the learned representations inherit the structures in the original data, graph-regularization terms are often added to the loss function. However, the prior graph construction often fails to reflect the true network connectivity and the intrinsic relationships. In addition, many graph-regularized methods fail to take the dual spaces into account. Probabilistic models are often used to model the distribution of the representations, but most of previous methods often assume that the hidden variables are independent and identically distributed for simplicity. To this end, we propose a learnable graph-regularization model for matrix decomposition (LGMD), which builds a bridge between graph-regularized methods and probabilistic matrix decomposition models. LGMD learns two graphical structures (i.e., two precision matrices) in real-time in an iterative manner via sparse precision matrix estimation and is more robust to noise and missing entries. Extensive numerical results and comparison with competing methods demonstrate its effectiveness.

Related papers

• A Complete Decomposition of KL Error using Refined Information and Mode Interaction Selection [11.994525728378603]
  We revisit the classical formulation of the log-linear model with a focus on higher-order mode interactions. We find that our learned distributions are able to more efficiently use the finite amount of data which is available in practice.
  arXiv  Detail & Related papers  (2024-10-15T18:08:32Z)
• Dissecting embedding method: learning higher-order structures from data [0.0]
  Geometric deep learning methods for data learning often include set of assumptions on the geometry of the feature space. These assumptions together with data being discrete and finite can cause some generalisations, which are likely to create wrong interpretations of the data and models outputs.
  arXiv  Detail & Related papers  (2024-10-14T08:19:39Z)
• Learning Graphical Factor Models with Riemannian Optimization [70.13748170371889]
  This paper proposes a flexible algorithmic framework for graph learning under low-rank structural constraints. The problem is expressed as penalized maximum likelihood estimation of an elliptical distribution. We leverage geometries of positive definite matrices and positive semi-definite matrices of fixed rank that are well suited to elliptical models.
  arXiv  Detail & Related papers  (2022-10-21T13:19:45Z)
• Graph Polynomial Convolution Models for Node Classification of Non-Homophilous Graphs [52.52570805621925]
  We investigate efficient learning from higher-order graph convolution and learning directly from adjacency matrix for node classification. We show that the resulting model lead to new graphs and residual scaling parameter. We demonstrate that the proposed methods obtain improved accuracy for node-classification of non-homophilous parameters.
  arXiv  Detail & Related papers  (2022-09-12T04:46:55Z)
• uGLAD: Sparse graph recovery by optimizing deep unrolled networks [11.48281545083889]
  We present a novel technique to perform sparse graph recovery by optimizing deep unrolled networks. Our model, uGLAD, builds upon and extends the state-of-the-art model GLAD to the unsupervised setting. We evaluate model results on synthetic Gaussian data, non-Gaussian data generated from Gene Regulatory Networks, and present a case study in anaerobic digestion.
  arXiv  Detail & Related papers  (2022-05-23T20:20:27Z)
• Learning Time-Varying Graphs from Online Data [39.21234914444073]
  This work proposes an algorithmic framework to learn time-varying graphs from online data. It renders it model-independent, i.e., it can be theoretically analyzed in its abstract formulation. We specialize the framework to three well-known graph learning models, namely, the Gaussian graphical model (GGM), the structural equation model (SEM), and the smoothness-based model (SBM)
  arXiv  Detail & Related papers  (2021-10-21T09:46:44Z)
• Distributed Training of Graph Convolutional Networks using Subgraph Approximation [72.89940126490715]
  We propose a training strategy that mitigates the lost information across multiple partitions of a graph through a subgraph approximation scheme. The subgraph approximation approach helps the distributed training system converge at single-machine accuracy.
  arXiv  Detail & Related papers  (2020-12-09T09:23:49Z)
• Multilayer Clustered Graph Learning [66.94201299553336]
  We use contrastive loss as a data fidelity term, in order to properly aggregate the observed layers into a representative graph. Experiments show that our method leads to a clustered clusters w.r.t. We learn a clustering algorithm for solving clustering problems.
  arXiv  Detail & Related papers  (2020-10-29T09:58:02Z)
• Spectral Learning on Matrices and Tensors [74.88243719463053]
  We show that tensor decomposition can pick up latent effects that are missed by matrix methods. We also outline computational techniques to design efficient tensor decomposition methods.
  arXiv  Detail & Related papers  (2020-04-16T22:53:00Z)
• Block-Approximated Exponential Random Graphs [77.4792558024487]
  An important challenge in the field of exponential random graphs (ERGs) is the fitting of non-trivial ERGs on large graphs. We propose an approximative framework to such non-trivial ERGs that result in dyadic independence (i.e., edge independent) distributions. Our methods are scalable to sparse graphs consisting of millions of nodes.
  arXiv  Detail & Related papers  (2020-02-14T11:42:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.