Disambiguate First Parse Later: Generating Interpretations for Ambiguity Resolution in Semantic Parsing

• URL: http://arxiv.org/abs/2502.18448v1
• Date: Tue, 25 Feb 2025 18:42:26 GMT
• Title: Disambiguate First Parse Later: Generating Interpretations for Ambiguity Resolution in Semantic Parsing
• Authors: Irina Saparina, Mirella Lapata,
• Abstract summary: We propose a modular approach that resolves ambiguity using natural language interpretations before mapping these to logical forms.<n>Our approach improves interpretation coverage and generalizes across datasets with different annotation styles, database structures, and ambiguity types.
• Score: 56.82807063333088
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Handling ambiguity and underspecification is an important challenge in natural language interfaces, particularly for tasks like text-to-SQL semantic parsing. We propose a modular approach that resolves ambiguity using natural language interpretations before mapping these to logical forms (e.g., SQL queries). Although LLMs excel at parsing unambiguous utterances, they show strong biases for ambiguous ones, typically predicting only preferred interpretations. We constructively exploit this bias to generate an initial set of preferred disambiguations and then apply a specialized infilling model to identify and generate missing interpretations. To train the infilling model, we introduce an annotation method that uses SQL execution to validate different meanings. Our approach improves interpretation coverage and generalizes across datasets with different annotation styles, database structures, and ambiguity types.

Related papers

• H-STAR: LLM-driven Hybrid SQL-Text Adaptive Reasoning on Tables [56.73919743039263]
  This paper introduces a novel algorithm that integrates both symbolic and semantic (textual) approaches in a two-stage process to address limitations. Our experiments demonstrate that H-STAR significantly outperforms state-of-the-art methods across three question-answering (QA) and fact-verification datasets.
  arXiv  Detail & Related papers  (2024-06-29T21:24:19Z)
• AMBROSIA: A Benchmark for Parsing Ambiguous Questions into Database Queries [56.82807063333088]
  We introduce a new benchmark, AMBROSIA, which we hope will inform and inspire the development of text-to-open programs. Our dataset contains questions showcasing three different types of ambiguity (scope ambiguity, attachment ambiguity, and vagueness) In each case, the ambiguity persists even when the database context is provided. This is achieved through a novel approach that involves controlled generation of databases from scratch.
  arXiv  Detail & Related papers  (2024-06-27T10:43:04Z)
• Zero and Few-shot Semantic Parsing with Ambiguous Inputs [45.285508941560295]
  We introduce AmP, a framework, dataset, and challenge for translating ambiguous natural language to formal representations like logic and code. Using AmP, we investigate how several few-shot text-to-code systems handle ambiguity, introducing three new metrics. We find that large pre-trained models perform poorly at capturing the distribution of possible meanings without deliberate instruction.
  arXiv  Detail & Related papers  (2023-06-01T15:46:36Z)
• Interpretable Word Sense Representations via Definition Generation: The Case of Semantic Change Analysis [3.515619810213763]
  We propose using automatically generated natural language definitions of contextualised word usages as interpretable word and word sense representations. We demonstrate how the resulting sense labels can make existing approaches to semantic change analysis more interpretable.
  arXiv  Detail & Related papers  (2023-05-19T20:36:21Z)
• Design Choices for Crowdsourcing Implicit Discourse Relations: Revealing the Biases Introduced by Task Design [23.632204469647526]
  We show that the task design can push annotators towards certain relations. We conclude that this type of bias should be taken into account when training and testing models.
  arXiv  Detail & Related papers  (2023-04-03T09:04:18Z)
• SUN: Exploring Intrinsic Uncertainties in Text-to-SQL Parsers [61.48159785138462]
  This paper aims to improve the performance of text-to-dependence by exploring the intrinsic uncertainties in the neural network based approaches (called SUN) Extensive experiments on five benchmark datasets demonstrate that our method significantly outperforms competitors and achieves new state-of-the-art results.
  arXiv  Detail & Related papers  (2022-09-14T06:27:51Z)
• Two Contrasting Data Annotation Paradigms for Subjective NLP Tasks [17.033055327465238]
  We propose two contrasting paradigms for data annotation. The descriptive paradigm encourages annotator subjectivity, whereas the prescriptive paradigm discourages it. We argue that dataset creators should explicitly aim for one or the other to facilitate the intended use of their dataset.
  arXiv  Detail & Related papers  (2021-12-14T15:38:22Z)
• On The Ingredients of an Effective Zero-shot Semantic Parser [95.01623036661468]
  We analyze zero-shot learning by paraphrasing training examples of canonical utterances and programs from a grammar. We propose bridging these gaps using improved grammars, stronger paraphrasers, and efficient learning methods. Our model achieves strong performance on two semantic parsing benchmarks (Scholar, Geo) with zero labeled data.
  arXiv  Detail & Related papers  (2021-10-15T21:41:16Z)
• Infusing Finetuning with Semantic Dependencies [62.37697048781823]
  We show that, unlike syntax, semantics is not brought to the surface by today's pretrained models. We then use convolutional graph encoders to explicitly incorporate semantic parses into task-specific finetuning.
  arXiv  Detail & Related papers  (2020-12-10T01:27:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.