Probabilistic Embeddings with Laplacian Graph Priors
- URL: http://arxiv.org/abs/2204.01846v1
- Date: Fri, 25 Mar 2022 13:33:51 GMT
- Title: Probabilistic Embeddings with Laplacian Graph Priors
- Authors: V\"ain\"o Yrj\"an\"ainen and M{\aa}ns Magnusson
- Abstract summary: We show that the model unifies several previously proposed embedding methods under one umbrella.
We empirically show that our model matches the performance of previous models as special cases.
We provide code as an implementation enabling flexible estimation in different settings.
- Score: 0.0
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: We introduce probabilistic embeddings using Laplacian priors (PELP). The
proposed model enables incorporating graph side-information into static word
embeddings. We theoretically show that the model unifies several previously
proposed embedding methods under one umbrella. PELP generalises graph-enhanced,
group, dynamic, and cross-lingual static word embeddings. PELP also enables any
combination of these previous models in a straightforward fashion. Furthermore,
we empirically show that our model matches the performance of previous models
as special cases. In addition, we demonstrate its flexibility by applying it to
the comparison of political sociolects over time. Finally, we provide code as a
TensorFlow implementation enabling flexible estimation in different settings.
Related papers
- Meaning Representations from Trajectories in Autoregressive Models [106.63181745054571]
We propose to extract meaning representations from autoregressive language models by considering the distribution of all possible trajectories extending an input text.
This strategy is prompt-free, does not require fine-tuning, and is applicable to any pre-trained autoregressive model.
We empirically show that the representations obtained from large models align well with human annotations, outperform other zero-shot and prompt-free methods on semantic similarity tasks, and can be used to solve more complex entailment and containment tasks that standard embeddings cannot handle.
arXiv Detail & Related papers (2023-10-23T04:35:58Z) - Word Embedding with Neural Probabilistic Prior [24.893999575628452]
We propose a probabilistic prior which can be seamlessly integrated with word embedding models.
The structure of the proposed prior is simple and effective, and it can be easily implemented and flexibly plugged in.
arXiv Detail & Related papers (2023-09-21T06:54:32Z) - Modeling Edge Features with Deep Bayesian Graph Networks [23.32339964726699]
We introduce an additional Bayesian network mapping edge features into discrete states to be used by the original model.
By keeping the computational complexity linear in the number of edges, the proposed model is amenable to large-scale graph processing.
arXiv Detail & Related papers (2023-08-17T16:29:17Z) - Beyond Conjugacy for Chain Event Graph Model Selection [0.0]
Chain event graphs are a family of probabilistic graphical models that generalise Bayesian networks.
We propose a mixture modelling approach to model selection in chain event graphs that does not rely on conjugacy.
arXiv Detail & Related papers (2022-11-07T10:33:01Z) - Ordinal Graph Gamma Belief Network for Social Recommender Systems [54.9487910312535]
We develop a hierarchical Bayesian model termed ordinal graph factor analysis (OGFA), which jointly models user-item and user-user interactions.
OGFA not only achieves good recommendation performance, but also extracts interpretable latent factors corresponding to representative user preferences.
We extend OGFA to ordinal graph gamma belief network, which is a multi-stochastic-layer deep probabilistic model.
arXiv Detail & Related papers (2022-09-12T09:19:22Z) - Entropy-based Characterization of Modeling Constraints [0.0]
In most data-scientific approaches, the principle of Entropy (MaxEnt) is used to justify some parametric model.
We derive the distribution over all viable distributions that satisfy the provided set of constraints.
The appropriate parametric model which is supported by the data can be always deduced at the end of model selection.
arXiv Detail & Related papers (2022-06-27T17:25:49Z) - A Graph-Enhanced Click Model for Web Search [67.27218481132185]
We propose a novel graph-enhanced click model (GraphCM) for web search.
We exploit both intra-session and inter-session information for the sparsity and cold-start problems.
arXiv Detail & Related papers (2022-06-17T08:32:43Z) - Combining Diverse Feature Priors [90.74601233745047]
We show that models trained with diverse sets of feature priors have less overlapping failure modes.
We also demonstrate that jointly training such models on additional (unlabeled) data allows them to correct each other's mistakes.
arXiv Detail & Related papers (2021-10-15T17:31:10Z) - PSD Representations for Effective Probability Models [117.35298398434628]
We show that a recently proposed class of positive semi-definite (PSD) models for non-negative functions is particularly suited to this end.
We characterize both approximation and generalization capabilities of PSD models, showing that they enjoy strong theoretical guarantees.
Our results open the way to applications of PSD models to density estimation, decision theory and inference.
arXiv Detail & Related papers (2021-06-30T15:13:39Z) - Bayesian hierarchical stacking [10.371079239965836]
We show that stacking is most effective when the model predictive performance is heterogeneous in inputs.
With the input-varying yet partially-pooled model weights, hierarchical stacking improves average and conditional predictions.
arXiv Detail & Related papers (2021-01-22T05:19:49Z) - From Static to Dynamic Node Embeddings [61.58641072424504]
We introduce a general framework for leveraging graph stream data for temporal prediction-based applications.
Our proposed framework includes novel methods for learning an appropriate graph time-series representation.
We find that the top-3 temporal models are always those that leverage the new $epsilon$-graph time-series representation.
arXiv Detail & Related papers (2020-09-21T16:48:29Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.