Weakly Supervised Semantic Parsing with Execution-based Spurious Program Filtering

• URL: http://arxiv.org/abs/2311.01161v1
• Date: Thu, 2 Nov 2023 11:45:40 GMT
• Title: Weakly Supervised Semantic Parsing with Execution-based Spurious Program Filtering
• Authors: Kang-il Lee, Segwang Kim, Kyomin Jung
• Abstract summary: We propose a domain-agnostic filtering mechanism based on program execution results. We run a majority vote on these representations to identify and filter out programs with significantly different semantics from the other programs.
• Score: 19.96076749160955
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The problem of spurious programs is a longstanding challenge when training a semantic parser from weak supervision. To eliminate such programs that have wrong semantics but correct denotation, existing methods focus on exploiting similarities between examples based on domain-specific knowledge. In this paper, we propose a domain-agnostic filtering mechanism based on program execution results. Specifically, for each program obtained through the search process, we first construct a representation that captures the program's semantics as execution results under various inputs. Then, we run a majority vote on these representations to identify and filter out programs with significantly different semantics from the other programs. In particular, our method is orthogonal to the program search process so that it can easily augment any of the existing weakly supervised semantic parsing frameworks. Empirical evaluations on the Natural Language Visual Reasoning and WikiTableQuestions demonstrate that applying our method to the existing semantic parsers induces significantly improved performances.

Related papers

• De-fine: Decomposing and Refining Visual Programs with Auto-Feedback [81.08213203440634]
  We introduce De-fine, a framework that decomposes complex tasks into simpler subtasks and refines programs through auto-feedback. Our experiments across various visual tasks show that De-fine creates more accurate and robust programs, setting new benchmarks in the field.
  arXiv  Detail & Related papers  (2023-11-21T06:24:09Z)
• InvAASTCluster: On Applying Invariant-Based Program Clustering to Introductory Programming Assignments [0.0]
  InvAASTCluster is a novel approach for program clustering. It takes advantage of dynamically generated program invariants observed over several program executions to cluster semantically equivalent IPAs. Our results show that InvAASTCluster advances the current state-of-the-art when used by clustering-based program repair tools.
  arXiv  Detail & Related papers  (2022-06-28T17:42:28Z)
• Natural Language to Code Translation with Execution [82.52142893010563]
  Execution result--minimum Bayes risk decoding for program selection. We show that it improves the few-shot performance of pretrained code models on natural-language-to-code tasks.
  arXiv  Detail & Related papers  (2022-04-25T06:06:08Z)
• Searching for More Efficient Dynamic Programs [61.79535031840558]
  We describe a set of program transformations, a simple metric for assessing the efficiency of a transformed program, and a search procedure to improve this metric. We show that in practice, automated search can find substantial improvements to the initial program.
  arXiv  Detail & Related papers  (2021-09-14T20:52:55Z)
• Enforcing Consistency in Weakly Supervised Semantic Parsing [68.2211621631765]
  We explore the use of consistency between the output programs for related inputs to reduce the impact of spurious programs. We find that a more consistent formalism leads to improved model performance even without consistency-based training.
  arXiv  Detail & Related papers  (2021-07-13T03:48:04Z)
• Learning from Executions for Semantic Parsing [86.94309120789396]
  We focus on the task of semi-supervised learning where a limited amount of annotated data is available. We propose to encourage executable programs for unlabeled utterances.
  arXiv  Detail & Related papers  (2021-04-12T21:07:53Z)
• Representing Partial Programs with Blended Abstract Semantics [62.20775388513027]
  We introduce a technique for representing partially written programs in a program synthesis engine. We learn an approximate execution model implemented as a modular neural network. We show that these hybrid neuro-symbolic representations enable execution-guided synthesizers to use more powerful language constructs.
  arXiv  Detail & Related papers  (2020-12-23T20:40:18Z)
• Learning Differentiable Programs with Admissible Neural Heuristics [43.54820901841979]
  We study the problem of learning differentiable functions expressed as programs in a domain-specific language. We frame this optimization problem as a search in a weighted graph whose paths encode top-down derivations of program syntax. Our key innovation is to view various classes of neural networks as continuous relaxations over the space of programs.
  arXiv  Detail & Related papers  (2020-07-23T16:07:39Z)
• Bringing freedom in variable choice when searching counter-examples in floating point programs [0.5161531917413706]
  This paper focuses on search strategies for CSPs using floating point numbers constraint systems. It introduces a new search based on the global number of occurrences that outperforms state-of-the-art strategies.
  arXiv  Detail & Related papers  (2020-02-27T21:20:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.