Probabilistic Embeddings with Laplacian Graph Priors

• URL: http://arxiv.org/abs/2204.01846v1
• Date: Fri, 25 Mar 2022 13:33:51 GMT
• Title: Probabilistic Embeddings with Laplacian Graph Priors
• Authors: V\"ain\"o Yrj\"an\"ainen and M{\aa}ns Magnusson
• Abstract summary: We show that the model unifies several previously proposed embedding methods under one umbrella. We empirically show that our model matches the performance of previous models as special cases. We provide code as an implementation enabling flexible estimation in different settings.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce probabilistic embeddings using Laplacian priors (PELP). The proposed model enables incorporating graph side-information into static word embeddings. We theoretically show that the model unifies several previously proposed embedding methods under one umbrella. PELP generalises graph-enhanced, group, dynamic, and cross-lingual static word embeddings. PELP also enables any combination of these previous models in a straightforward fashion. Furthermore, we empirically show that our model matches the performance of previous models as special cases. In addition, we demonstrate its flexibility by applying it to the comparison of political sociolects over time. Finally, we provide code as a TensorFlow implementation enabling flexible estimation in different settings.

Related papers

• Graph-Dictionary Signal Model for Sparse Representations of Multivariate Data [49.77103348208835]
  We define a novel Graph-Dictionary signal model, where a finite set of graphs characterizes relationships in data distribution through a weighted sum of their Laplacians. We propose a framework to infer the graph dictionary representation from observed data, along with a bilinear generalization of the primal-dual splitting algorithm to solve the learning problem. We exploit graph-dictionary representations in a motor imagery decoding task on brain activity data, where we classify imagined motion better than standard methods.
  arXiv  Detail & Related papers  (2024-11-08T17:40:43Z)
• Hierarchical mixtures of Unigram models for short text clustering: the role of Beta-Liouville priors [1.03590082373586]
  This paper presents a variant of the Multinomial mixture model tailored for the unsupervised classification of short text data. We explore an alternative priorational--the Beta-Liouville distribution--which offers a more flexible correlation structure than the Dirichlet.
  arXiv  Detail & Related papers  (2024-10-29T08:56:29Z)
• Hierarchical Blockmodelling for Knowledge Graphs [0.5530212768657544]
  We use blockmodels for the purpose of hierarchical entity clustering on knowledge graphs. The integration of the Nested Chinese Restaurant Process and the Stick Breaking Process into the generative model allows for the induction of hierarchical clusterings. We evaluate our model on synthetic and real-world datasets and quantitatively compare against benchmark models.
  arXiv  Detail & Related papers  (2024-08-28T09:04:15Z)
• Meaning Representations from Trajectories in Autoregressive Models [106.63181745054571]
  We propose to extract meaning representations from autoregressive language models by considering the distribution of all possible trajectories extending an input text. This strategy is prompt-free, does not require fine-tuning, and is applicable to any pre-trained autoregressive model. We empirically show that the representations obtained from large models align well with human annotations, outperform other zero-shot and prompt-free methods on semantic similarity tasks, and can be used to solve more complex entailment and containment tasks that standard embeddings cannot handle.
  arXiv  Detail & Related papers  (2023-10-23T04:35:58Z)
• Word Embedding with Neural Probabilistic Prior [24.893999575628452]
  We propose a probabilistic prior which can be seamlessly integrated with word embedding models. The structure of the proposed prior is simple and effective, and it can be easily implemented and flexibly plugged in.
  arXiv  Detail & Related papers  (2023-09-21T06:54:32Z)
• Beyond Conjugacy for Chain Event Graph Model Selection [0.0]
  Chain event graphs are a family of probabilistic graphical models that generalise Bayesian networks. We propose a mixture modelling approach to model selection in chain event graphs that does not rely on conjugacy.
  arXiv  Detail & Related papers  (2022-11-07T10:33:01Z)
• Ordinal Graph Gamma Belief Network for Social Recommender Systems [54.9487910312535]
  We develop a hierarchical Bayesian model termed ordinal graph factor analysis (OGFA), which jointly models user-item and user-user interactions. OGFA not only achieves good recommendation performance, but also extracts interpretable latent factors corresponding to representative user preferences. We extend OGFA to ordinal graph gamma belief network, which is a multi-stochastic-layer deep probabilistic model.
  arXiv  Detail & Related papers  (2022-09-12T09:19:22Z)
• A Graph-Enhanced Click Model for Web Search [67.27218481132185]
  We propose a novel graph-enhanced click model (GraphCM) for web search. We exploit both intra-session and inter-session information for the sparsity and cold-start problems.
  arXiv  Detail & Related papers  (2022-06-17T08:32:43Z)
• Combining Diverse Feature Priors [90.74601233745047]
  We show that models trained with diverse sets of feature priors have less overlapping failure modes. We also demonstrate that jointly training such models on additional (unlabeled) data allows them to correct each other's mistakes.
  arXiv  Detail & Related papers  (2021-10-15T17:31:10Z)
• PSD Representations for Effective Probability Models [117.35298398434628]
  We show that a recently proposed class of positive semi-definite (PSD) models for non-negative functions is particularly suited to this end. We characterize both approximation and generalization capabilities of PSD models, showing that they enjoy strong theoretical guarantees. Our results open the way to applications of PSD models to density estimation, decision theory and inference.
  arXiv  Detail & Related papers  (2021-06-30T15:13:39Z)
• Bayesian hierarchical stacking [10.371079239965836]
  We show that stacking is most effective when the model predictive performance is heterogeneous in inputs. With the input-varying yet partially-pooled model weights, hierarchical stacking improves average and conditional predictions.
  arXiv  Detail & Related papers  (2021-01-22T05:19:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.