MetaSRL++: A Uniform Scheme for Modelling Deeper Semantics

• URL: http://arxiv.org/abs/2305.09534v1
• Date: Tue, 16 May 2023 15:26:52 GMT
• Title: MetaSRL++: A Uniform Scheme for Modelling Deeper Semantics
• Authors: Fritz Hohl, Nianheng Wu, Martina Galetti, Remi van Trijp
• Abstract summary: This paper argues that in order to arrive at such a scheme, we also need a common modelling scheme. It introduces MetaSRL++, a uniform, language- and modality-independent modelling scheme based on Semantic Graphs.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Despite enormous progress in Natural Language Processing (NLP), our field is still lacking a common deep semantic representation scheme. As a result, the problem of meaning and understanding is typically sidestepped through more simple, approximative methods. This paper argues that in order to arrive at such a scheme, we also need a common modelling scheme. It therefore introduces MetaSRL++, a uniform, language- and modality-independent modelling scheme based on Semantic Graphs, as a step towards a common representation scheme; as well as a method for defining the concepts and entities that are used in these graphs. Our output is twofold. First, we illustrate MetaSRL++ through concrete examples. Secondly, we discuss how it relates to existing work in the field.

Related papers

• What Makes Two Language Models Think Alike? [6.244579327420724]
  We propose a new approach, based on metric-learning encoding models (MLEMs), as a first step to answer this question. MLEMs offer a transparent comparison, by identifying the specific linguistic features responsible for similarities and differences. The approach can straightforwardly be extended to other domains, such as speech and vision, and to other neural systems, including human brains.
  arXiv  Detail & Related papers  (2024-06-18T13:45:50Z)
• The Geometry of Categorical and Hierarchical Concepts in Large Language Models [15.126806053878855]
  We show how to extend the formalization of the linear representation hypothesis to represent features (e.g., is_animal) as vectors. We use the formalization to prove a relationship between the hierarchical structure of concepts and the geometry of their representations. We validate these theoretical results on the Gemma and LLaMA-3 large language models, estimating representations for 900+ hierarchically related concepts using data from WordNet.
  arXiv  Detail & Related papers  (2024-06-03T16:34:01Z)
• On the Origins of Linear Representations in Large Language Models [51.88404605700344]
  We introduce a simple latent variable model to formalize the concept dynamics of the next token prediction. Experiments show that linear representations emerge when learning from data matching the latent variable model. We additionally confirm some predictions of the theory using the LLaMA-2 large language model.
  arXiv  Detail & Related papers  (2024-03-06T17:17:36Z)
• Meaning Representations from Trajectories in Autoregressive Models [106.63181745054571]
  We propose to extract meaning representations from autoregressive language models by considering the distribution of all possible trajectories extending an input text. This strategy is prompt-free, does not require fine-tuning, and is applicable to any pre-trained autoregressive model. We empirically show that the representations obtained from large models align well with human annotations, outperform other zero-shot and prompt-free methods on semantic similarity tasks, and can be used to solve more complex entailment and containment tasks that standard embeddings cannot handle.
  arXiv  Detail & Related papers  (2023-10-23T04:35:58Z)
• Concept Algebra for (Score-Based) Text-Controlled Generative Models [27.725860408234478]
  This paper concerns the structure of learned representations in text-guided generative models. A key property of such models is that they can compose disparate concepts in a disentangled' manner. Here, we focus on the idea that concepts are encoded as subspaces of some representation space.
  arXiv  Detail & Related papers  (2023-02-07T20:43:48Z)
• Semantic Role Labeling Meets Definition Modeling: Using Natural Language to Describe Predicate-Argument Structures [104.32063681736349]
  We present an approach to describe predicate-argument structures using natural language definitions instead of discrete labels. Our experiments and analyses on PropBank-style and FrameNet-style, dependency-based and span-based SRL also demonstrate that a flexible model with an interpretable output does not necessarily come at the expense of performance.
  arXiv  Detail & Related papers  (2022-12-02T11:19:16Z)
• Masked Part-Of-Speech Model: Does Modeling Long Context Help Unsupervised POS-tagging? [94.68962249604749]
  We propose a Masked Part-of-Speech Model (MPoSM) to facilitate flexible dependency modeling. MPoSM can model arbitrary tag dependency and perform POS induction through the objective of masked POS reconstruction. We achieve competitive results on both the English Penn WSJ dataset and the universal treebank containing 10 diverse languages.
  arXiv  Detail & Related papers  (2022-06-30T01:43:05Z)
• Few-Shot Semantic Parsing with Language Models Trained On Code [52.23355024995237]
  We find that Codex performs better at semantic parsing than equivalent GPT-3 models. We find that unlike GPT-3, Codex performs similarly when targeting meaning representations directly, perhaps as meaning representations used in semantic parsing are structured similar to code.
  arXiv  Detail & Related papers  (2021-12-16T08:34:06Z)
• Definitions and Semantic Simulations Based on Object-Oriented Analysis and Modeling [0.0]
  We have proposed going beyond traditional to use rich semantics implemented in programming languages for modeling. In this paper, we discuss the application of executable models to two examples, first a structured definition of a waterfall and the cardiopulmonary system.
  arXiv  Detail & Related papers  (2019-12-31T05:59:02Z)
• A Simple Joint Model for Improved Contextual Neural Lemmatization [60.802451210656805]
  We present a simple joint neural model for lemmatization and morphological tagging that achieves state-of-the-art results on 20 languages. Our paper describes the model in addition to training and decoding procedures.
  arXiv  Detail & Related papers  (2019-04-04T02:03:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.