Enforcing Consistency in Weakly Supervised Semantic Parsing
- URL: http://arxiv.org/abs/2107.05833v1
- Date: Tue, 13 Jul 2021 03:48:04 GMT
- Title: Enforcing Consistency in Weakly Supervised Semantic Parsing
- Authors: Nitish Gupta, Sameer Singh, Matt Gardner
- Abstract summary: We explore the use of consistency between the output programs for related inputs to reduce the impact of spurious programs.
We find that a more consistent formalism leads to improved model performance even without consistency-based training.
- Score: 68.2211621631765
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: The predominant challenge in weakly supervised semantic parsing is that of
spurious programs that evaluate to correct answers for the wrong reasons. Prior
work uses elaborate search strategies to mitigate the prevalence of spurious
programs; however, they typically consider only one input at a time. In this
work we explore the use of consistency between the output programs for related
inputs to reduce the impact of spurious programs. We bias the program search
(and thus the model's training signal) towards programs that map the same
phrase in related inputs to the same sub-parts in their respective programs.
Additionally, we study the importance of designing logical formalisms that
facilitate this kind of consAistency-based training. We find that a more
consistent formalism leads to improved model performance even without
consistency-based training. When combined together, these two insights lead to
a 10% absolute improvement over the best prior result on the Natural Language
Visual Reasoning dataset.
Related papers
- Weakly Supervised Semantic Parsing with Execution-based Spurious Program
Filtering [19.96076749160955]
We propose a domain-agnostic filtering mechanism based on program execution results.
We run a majority vote on these representations to identify and filter out programs with significantly different semantics from the other programs.
arXiv Detail & Related papers (2023-11-02T11:45:40Z) - Improving Language Models Meaning Understanding and Consistency by
Learning Conceptual Roles from Dictionary [65.268245109828]
Non-human-like behaviour of contemporary pre-trained language models (PLMs) is a leading cause undermining their trustworthiness.
A striking phenomenon is the generation of inconsistent predictions, which produces contradictory results.
We propose a practical approach that alleviates the inconsistent behaviour issue by improving PLM awareness.
arXiv Detail & Related papers (2023-10-24T06:15:15Z) - Learning from Self-Sampled Correct and Partially-Correct Programs [96.66452896657991]
We propose to let the model perform sampling during training and learn from both self-sampled fully-correct programs and partially-correct programs.
We show that our use of self-sampled correct and partially-correct programs can benefit learning and help guide the sampling process.
Our proposed method improves the pass@k performance by 3.1% to 12.3% compared to learning from a single reference program with MLE.
arXiv Detail & Related papers (2022-05-28T03:31:07Z) - On the Configuration of More and Less Expressive Logic Programs [11.331373810571993]
We consider two well-known model-based AI methodologies, SAT and ASP, define a number of syntactic features that may characterise their inputs.
Results of a wide experimental analysis involving SAT and ASP domains, taken from respective competitions, show the different advantages that can be obtained by using input reformulation and configuration.
arXiv Detail & Related papers (2022-03-02T10:55:35Z) - Learning from Executions for Semantic Parsing [86.94309120789396]
We focus on the task of semi-supervised learning where a limited amount of annotated data is available.
We propose to encourage executable programs for unlabeled utterances.
arXiv Detail & Related papers (2021-04-12T21:07:53Z) - Latent Programmer: Discrete Latent Codes for Program Synthesis [56.37993487589351]
In many sequence learning tasks, such as program synthesis and document summarization, a key problem is searching over a large space of possible output sequences.
We propose to learn representations of the outputs that are specifically meant for search: rich enough to specify the desired output but compact enough to make search more efficient.
We introduce the emphLatent Programmer, a program synthesis method that first predicts a discrete latent code from input/output examples, and then generates the program in the target language.
arXiv Detail & Related papers (2020-12-01T10:11:35Z) - Verifying Tight Logic Programs with anthem and Vampire [7.804960968120875]
We extend the definition of program completion to programs with input and output in a subset of the input language of the ASP grounder gringo.
We study the relationship between stable models and completion in this context, and describe preliminary experiments with the use of two software tools.
arXiv Detail & Related papers (2020-08-05T10:01:33Z) - BUSTLE: Bottom-Up Program Synthesis Through Learning-Guided Exploration [72.88493072196094]
We present a new synthesis approach that leverages learning to guide a bottom-up search over programs.
In particular, we train a model to prioritize compositions of intermediate values during search conditioned on a set of input-output examples.
We show that the combination of learning and bottom-up search is remarkably effective, even with simple supervised learning approaches.
arXiv Detail & Related papers (2020-07-28T17:46:18Z) - Program Synthesis with Pragmatic Communication [28.24612900419843]
This work introduces a new inductive bias derived by modeling the program synthesis task as rational communication.
A user study finds that end-user participants communicate more effectively with the pragmatic program synthesizer over a non-pragmatic one.
arXiv Detail & Related papers (2020-07-09T20:55:44Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.