Revisiting Generative Commonsense Reasoning: A Pre-Ordering Approach
- URL: http://arxiv.org/abs/2205.13183v1
- Date: Thu, 26 May 2022 06:36:53 GMT
- Title: Revisiting Generative Commonsense Reasoning: A Pre-Ordering Approach
- Authors: Chao Zhao and Faeze Brahman and Tenghao Huang and Snigdha Chaturvedi
- Abstract summary: We argue that the order of the input concepts can affect the PTM's ability to utilize its commonsense knowledge.
We propose a pre-ordering approach to elaborately manipulate the order of the given concepts before generation.
- Score: 16.91261958272558
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Pre-trained models (PTMs) have lead to great improvements in natural language
generation (NLG). However, it is still unclear how much commonsense knowledge
they possess. With the goal of evaluating commonsense knowledge of NLG models,
recent work has proposed the problem of generative commonsense reasoning, e.g.,
to compose a logical sentence given a set of unordered concepts. Existing
approaches to this problem hypothesize that PTMs lack sufficient parametric
knowledge for this task, which can be overcome by introducing external
knowledge or task-specific pre-training objectives. Different from this trend,
we argue that PTM's inherent ability for generative commonsense reasoning is
underestimated due to the order-agnostic property of its input. In particular,
we hypothesize that the order of the input concepts can affect the PTM's
ability to utilize its commonsense knowledge. To this end, we propose a
pre-ordering approach to elaborately manipulate the order of the given concepts
before generation. Experiments show that our approach can outperform the more
sophisticated models that have access to a lot of external data and resources.
Related papers
- Improving Language Models Meaning Understanding and Consistency by
Learning Conceptual Roles from Dictionary [65.268245109828]
Non-human-like behaviour of contemporary pre-trained language models (PLMs) is a leading cause undermining their trustworthiness.
A striking phenomenon is the generation of inconsistent predictions, which produces contradictory results.
We propose a practical approach that alleviates the inconsistent behaviour issue by improving PLM awareness.
arXiv Detail & Related papers (2023-10-24T06:15:15Z) - Learning to Predict Concept Ordering for Common Sense Generation [32.2052248473022]
We study the relationship between the ordering of the input concepts and the quality of the generated sentences.
We find that BART-large model consistently outperforms all other LMs considered in this study.
The larger GPT3-based large language models (LLMs) variants do not necessarily outperform much smaller LMs on this task.
arXiv Detail & Related papers (2023-09-12T16:27:18Z) - CARE: Certifiably Robust Learning with Reasoning via Variational
Inference [26.210129662748862]
We propose a certifiably robust learning with reasoning pipeline (CARE)
CARE achieves significantly higher certified robustness compared with the state-of-the-art baselines.
We additionally conducted different ablation studies to demonstrate the empirical robustness of CARE and the effectiveness of different knowledge integration.
arXiv Detail & Related papers (2022-09-12T07:15:52Z) - Evaluate Confidence Instead of Perplexity for Zero-shot Commonsense
Reasoning [85.1541170468617]
This paper reconsiders the nature of commonsense reasoning and proposes a novel commonsense reasoning metric, Non-Replacement Confidence (NRC)
Our proposed novel method boosts zero-shot performance on two commonsense reasoning benchmark datasets and further seven commonsense question-answering datasets.
arXiv Detail & Related papers (2022-08-23T14:42:14Z) - Principled Knowledge Extrapolation with GANs [92.62635018136476]
We study counterfactual synthesis from a new perspective of knowledge extrapolation.
We show that an adversarial game with a closed-form discriminator can be used to address the knowledge extrapolation problem.
Our method enjoys both elegant theoretical guarantees and superior performance in many scenarios.
arXiv Detail & Related papers (2022-05-21T08:39:42Z) - Great Truths are Always Simple: A Rather Simple Knowledge Encoder for
Enhancing the Commonsense Reasoning Capacity of Pre-Trained Models [89.98762327725112]
Commonsense reasoning in natural language is a desired ability of artificial intelligent systems.
For solving complex commonsense reasoning tasks, a typical solution is to enhance pre-trained language models(PTMs) with a knowledge-aware graph neural network(GNN) encoder.
Despite the effectiveness, these approaches are built on heavy architectures, and can't clearly explain how external knowledge resources improve the reasoning capacity of PTMs.
arXiv Detail & Related papers (2022-05-04T01:27:36Z) - ArT: All-round Thinker for Unsupervised Commonsense Question-Answering [54.068032948300655]
We propose an approach of All-round Thinker (ArT) by fully taking association during knowledge generating.
We evaluate it on three commonsense QA benchmarks: COPA, SocialIQA and SCT.
arXiv Detail & Related papers (2021-12-26T18:06:44Z) - Logic Constraints to Feature Importances [17.234442722611803]
"Black box" nature of AI models is often a limit for a reliable application in high-stakes fields like diagnostic techniques, autonomous guide, etc.
Recent works have shown that an adequate level of interpretability could enforce the more general concept of model trustworthiness.
The basic idea of this paper is to exploit the human prior knowledge of the features' importance for a specific task, in order to coherently aid the phase of the model's fitting.
arXiv Detail & Related papers (2021-10-13T09:28:38Z) - Pre-Trained Models: Past, Present and Future [126.21572378910746]
Large-scale pre-trained models (PTMs) have recently achieved great success and become a milestone in the field of artificial intelligence (AI)
By storing knowledge into huge parameters and fine-tuning on specific tasks, the rich knowledge implicitly encoded in huge parameters can benefit a variety of downstream tasks.
It is now the consensus of the AI community to adopt PTMs as backbone for downstream tasks rather than learning models from scratch.
arXiv Detail & Related papers (2021-06-14T02:40:32Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.