Differentiable Open-Ended Commonsense Reasoning
- URL: http://arxiv.org/abs/2010.14439v2
- Date: Sun, 6 Jun 2021 20:20:27 GMT
- Title: Differentiable Open-Ended Commonsense Reasoning
- Authors: Bill Yuchen Lin, Haitian Sun, Bhuwan Dhingra, Manzil Zaheer, Xiang
Ren, William W. Cohen
- Abstract summary: We study open-ended commonsense reasoning (OpenCSR) using as a resource only a corpus of commonsense facts written in natural language.
As an approach to OpenCSR, we propose DrFact, an efficient Differentiable model for multi-hop Reasoning over knowledge Facts.
- Score: 80.94997942571838
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Current commonsense reasoning research focuses on developing models that use
commonsense knowledge to answer multiple-choice questions. However, systems
designed to answer multiple-choice questions may not be useful in applications
that do not provide a small list of candidate answers to choose from. As a step
towards making commonsense reasoning research more realistic, we propose to
study open-ended commonsense reasoning (OpenCSR) -- the task of answering a
commonsense question without any pre-defined choices -- using as a resource
only a corpus of commonsense facts written in natural language. OpenCSR is
challenging due to a large decision space, and because many questions require
implicit multi-hop reasoning. As an approach to OpenCSR, we propose DrFact, an
efficient Differentiable model for multi-hop Reasoning over knowledge Facts. To
evaluate OpenCSR methods, we adapt several popular commonsense reasoning
benchmarks, and collect multiple new answers for each test question via
crowd-sourcing. Experiments show that DrFact outperforms strong baseline
methods by a large margin.
Related papers
- Differentiating Choices via Commonality for Multiple-Choice Question Answering [54.04315943420376]
Multiple-choice question answering can provide valuable clues for choosing the right answer.
Existing models often rank each choice separately, overlooking the context provided by other choices.
We propose a novel model by differentiating choices through identifying and eliminating their commonality, called DCQA.
arXiv Detail & Related papers (2024-08-21T12:05:21Z) - Open-ended Commonsense Reasoning with Unrestricted Answer Scope [47.14397700770702]
Open-ended Commonsense Reasoning is defined as solving a commonsense question without providing 1) a short list of answer candidates and 2) a pre-defined answer scope.
In this work, we leverage pre-trained language models to iteratively retrieve reasoning paths on the external knowledge base.
The reasoning paths can help to identify the most precise answer to the commonsense question.
arXiv Detail & Related papers (2023-10-18T02:45:54Z) - Getting MoRE out of Mixture of Language Model Reasoning Experts [71.61176122960464]
We propose a Mixture-of-Reasoning-Experts (MoRE) framework that ensembles diverse specialized language models.
We specialize the backbone language model with prompts optimized for different reasoning categories, including factual, multihop, mathematical, and commonsense reasoning.
Our human study confirms that presenting expert predictions and the answer selection process helps annotators more accurately calibrate when to trust the system's output.
arXiv Detail & Related papers (2023-05-24T02:00:51Z) - Multi-hop Commonsense Knowledge Injection Framework for Zero-Shot
Commonsense Question Answering [6.086719709100659]
We propose a novel multi-hop commonsense knowledge injection framework.
Our framework achieves state-of-art performance on five commonsense question answering benchmarks.
arXiv Detail & Related papers (2023-05-10T07:13:47Z) - Federated Prompting and Chain-of-Thought Reasoning for Improving LLMs
Answering [13.735277588793997]
We investigate how to enhance answer precision in frequently asked questions posed by distributed users using cloud-based Large Language Models (LLMs)
Our study focuses on a typical situations where users ask similar queries that involve identical mathematical reasoning steps and problem-solving procedures.
We propose to improve the distributed synonymous questions using Self-Consistency (SC) and Chain-of-Thought (CoT) techniques.
arXiv Detail & Related papers (2023-04-27T01:48:03Z) - A Graph-Guided Reasoning Approach for Open-ended Commonsense Question
Answering [21.61166185452341]
We propose a reasoner that constructs a question-dependent open knowledge graph based on retrieved supporting facts and employs a sequential subgraph reasoning process to predict the answer.
Experiments on two OpenCSR datasets show that the proposed model achieves great performance on benchmark OpenCSR datasets.
arXiv Detail & Related papers (2023-03-18T11:15:33Z) - TSGP: Two-Stage Generative Prompting for Unsupervised Commonsense
Question Answering [4.965306353273393]
Unsupervised commonsense question answering requires mining effective commonsense knowledge without the rely on the labeled task data.
We propose a two-stage prompt-based unsupervised commonsense question answering framework (TSGP)
Experimental results and analysis on three different commonsense reasoning tasks, CommonsenseQA, OpenBookQA, and SocialIQA, demonstrate that TSGP significantly improves the reasoning ability of language models in unsupervised settings.
arXiv Detail & Related papers (2022-11-24T10:19:24Z) - Multifaceted Improvements for Conversational Open-Domain Question
Answering [54.913313912927045]
We propose a framework with Multifaceted Improvements for Conversational open-domain Question Answering (MICQA)
Firstly, the proposed KL-divergence based regularization is able to lead to a better question understanding for retrieval and answer reading.
Second, the added post-ranker module can push more relevant passages to the top placements and be selected for reader with a two-aspect constrains.
Third, the well designed curriculum learning strategy effectively narrows the gap between the golden passage settings of training and inference, and encourages the reader to find true answer without the golden passage assistance.
arXiv Detail & Related papers (2022-04-01T07:54:27Z) - Knowledgeable Dialogue Reading Comprehension on Key Turns [84.1784903043884]
Multi-choice machine reading comprehension (MRC) requires models to choose the correct answer from candidate options given a passage and a question.
Our research focuses dialogue-based MRC, where the passages are multi-turn dialogues.
It suffers from two challenges, the answer selection decision is made without support of latently helpful commonsense, and the multi-turn context may hide considerable irrelevant information.
arXiv Detail & Related papers (2020-04-29T07:04:43Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.