Distributional Inclusion Hypothesis and Quantifications: Probing for Hypernymy in Functional Distributional Semantics

• URL: http://arxiv.org/abs/2309.08325v2
• Date: Sat, 10 Feb 2024 17:57:14 GMT
• Title: Distributional Inclusion Hypothesis and Quantifications: Probing for Hypernymy in Functional Distributional Semantics
• Authors: Chun Hei Lo, Wai Lam, Hong Cheng, and Guy Emerson
• Abstract summary: Functional Distributional Semantics (FDS) models the meaning of words by truth-conditional functions. We show that FDS models learn hypernymy on a restricted class of corpus that strictly follows the Distributional Inclusion Hypothesis (DIH)
• Score: 50.363809539842386
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Functional Distributional Semantics (FDS) models the meaning of words by truth-conditional functions. This provides a natural representation for hypernymy but no guarantee that it can be learnt when FDS models are trained on a corpus. In this paper, we probe into FDS models and study the representations learnt, drawing connections between quantifications, the Distributional Inclusion Hypothesis (DIH), and the variational-autoencoding objective of FDS model training. Using synthetic data sets, we reveal that FDS models learn hypernymy on a restricted class of corpus that strictly follows the DIH. We further introduce a training objective that both enables hypernymy learning under the reverse of the DIH and improves hypernymy detection from real corpora.

Related papers

• Constrained Diffusion Models via Dual Training [80.03953599062365]
  We develop constrained diffusion models based on desired distributions informed by requirements. We show that our constrained diffusion models generate new data from a mixture data distribution that achieves the optimal trade-off among objective and constraints.
  arXiv  Detail & Related papers  (2024-08-27T14:25:42Z)
• Unveil Conditional Diffusion Models with Classifier-free Guidance: A Sharp Statistical Theory [87.00653989457834]
  Conditional diffusion models serve as the foundation of modern image synthesis and find extensive application in fields like computational biology and reinforcement learning. Despite the empirical success, theory of conditional diffusion models is largely missing. This paper bridges the gap by presenting a sharp statistical theory of distribution estimation using conditional diffusion models.
  arXiv  Detail & Related papers  (2024-03-18T17:08:24Z)
• On Memorization in Diffusion Models [46.656797890144105]
  We show that memorization behaviors tend to occur on smaller-sized datasets. We quantify the impact of the influential factors on these memorization behaviors in terms of effective model memorization (EMM) Our study holds practical significance for diffusion model users and offers clues to theoretical research in deep generative models.
  arXiv  Detail & Related papers  (2023-10-04T09:04:20Z)
• Training Data Protection with Compositional Diffusion Models [99.46239561159953]
  Compartmentalized Diffusion Models (CDM) are a method to train different diffusion models (or prompts) on distinct data sources. Individual models can be trained in isolation, at different times, and on different distributions and domains. Each model only contains information about a subset of the data it was exposed to during training, enabling several forms of training data protection.
  arXiv  Detail & Related papers  (2023-08-02T23:27:49Z)
• Diff-Instruct: A Universal Approach for Transferring Knowledge From Pre-trained Diffusion Models [77.83923746319498]
  We propose a framework called Diff-Instruct to instruct the training of arbitrary generative models. We show that Diff-Instruct results in state-of-the-art single-step diffusion-based models. Experiments on refining GAN models show that the Diff-Instruct can consistently improve the pre-trained generators of GAN models.
  arXiv  Detail & Related papers  (2023-05-29T04:22:57Z)
• Score-based Generative Modeling Through Backward Stochastic Differential Equations: Inversion and Generation [6.2255027793924285]
  The proposed BSDE-based diffusion model represents a novel approach to diffusion modeling, which extends the application of differential equations (SDEs) in machine learning. We demonstrate the theoretical guarantees of the model, the benefits of using Lipschitz networks for score matching, and its potential applications in various areas such as diffusion inversion, conditional diffusion, and uncertainty quantification.
  arXiv  Detail & Related papers  (2023-04-26T01:15:35Z)
• Towards Controllable Diffusion Models via Reward-Guided Exploration [15.857464051475294]
  We propose a novel framework that guides the training-phase of diffusion models via reinforcement learning (RL) RL enables calculating policy gradients via samples from a pay-off distribution proportional to exponential scaled rewards, rather than from policies themselves. Experiments on 3D shape and molecule generation tasks show significant improvements over existing conditional diffusion models.
  arXiv  Detail & Related papers  (2023-04-14T13:51:26Z)
• Flexible Amortized Variational Inference in qBOLD MRI [56.4324135502282]
  Oxygen extraction fraction (OEF) and deoxygenated blood volume (DBV) are more ambiguously determined from the data. Existing inference methods tend to yield very noisy and underestimated OEF maps, while overestimating DBV. This work describes a novel probabilistic machine learning approach that can infer plausible distributions of OEF and DBV.
  arXiv  Detail & Related papers  (2022-03-11T10:47:16Z)
• Parsimony-Enhanced Sparse Bayesian Learning for Robust Discovery of Partial Differential Equations [5.584060970507507]
  A Parsimony Enhanced Sparse Bayesian Learning (PeSBL) method is developed for discovering the governing Partial Differential Equations (PDEs) of nonlinear dynamical systems. Results of numerical case studies indicate that the governing PDEs of many canonical dynamical systems can be correctly identified using the proposed PeSBL method.
  arXiv  Detail & Related papers  (2021-07-08T00:56:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.