Open-ended Commonsense Reasoning with Unrestricted Answer Scope
- URL: http://arxiv.org/abs/2310.11672v2
- Date: Fri, 27 Oct 2023 13:50:09 GMT
- Title: Open-ended Commonsense Reasoning with Unrestricted Answer Scope
- Authors: Chen Ling, Xuchao Zhang, Xujiang Zhao, Yanchi Liu, Wei Cheng, Mika
Oishi, Takao Osaki, Katsushi Matsuda, Haifeng Chen, Liang Zhao
- Abstract summary: Open-ended Commonsense Reasoning is defined as solving a commonsense question without providing 1) a short list of answer candidates and 2) a pre-defined answer scope.
In this work, we leverage pre-trained language models to iteratively retrieve reasoning paths on the external knowledge base.
The reasoning paths can help to identify the most precise answer to the commonsense question.
- Score: 47.14397700770702
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Open-ended Commonsense Reasoning is defined as solving a commonsense question
without providing 1) a short list of answer candidates and 2) a pre-defined
answer scope. Conventional ways of formulating the commonsense question into a
question-answering form or utilizing external knowledge to learn
retrieval-based methods are less applicable in the open-ended setting due to an
inherent challenge. Without pre-defining an answer scope or a few candidates,
open-ended commonsense reasoning entails predicting answers by searching over
an extremely large searching space. Moreover, most questions require implicit
multi-hop reasoning, which presents even more challenges to our problem. In
this work, we leverage pre-trained language models to iteratively retrieve
reasoning paths on the external knowledge base, which does not require
task-specific supervision. The reasoning paths can help to identify the most
precise answer to the commonsense question. We conduct experiments on two
commonsense benchmark datasets. Compared to other approaches, our proposed
method achieves better performance both quantitatively and qualitatively.
Related papers
- Don't Just Say "I don't know"! Self-aligning Large Language Models for Responding to Unknown Questions with Explanations [70.6395572287422]
Self-alignment method is capable of not only refusing to answer but also providing explanation to the unanswerability of unknown questions.
We conduct disparity-driven self-curation to select qualified data for fine-tuning the LLM itself for aligning the responses to unknown questions as desired.
arXiv Detail & Related papers (2024-02-23T02:24:36Z) - Clarify When Necessary: Resolving Ambiguity Through Interaction with LMs [58.620269228776294]
We propose a task-agnostic framework for resolving ambiguity by asking users clarifying questions.
We evaluate systems across three NLP applications: question answering, machine translation and natural language inference.
We find that intent-sim is robust, demonstrating improvements across a wide range of NLP tasks and LMs.
arXiv Detail & Related papers (2023-11-16T00:18:50Z) - Answering Ambiguous Questions with a Database of Questions, Answers, and
Revisions [95.92276099234344]
We present a new state-of-the-art for answering ambiguous questions that exploits a database of unambiguous questions generated from Wikipedia.
Our method improves performance by 15% on recall measures and 10% on measures which evaluate disambiguating questions from predicted outputs.
arXiv Detail & Related papers (2023-08-16T20:23:16Z) - A Graph-Guided Reasoning Approach for Open-ended Commonsense Question
Answering [21.61166185452341]
We propose a reasoner that constructs a question-dependent open knowledge graph based on retrieved supporting facts and employs a sequential subgraph reasoning process to predict the answer.
Experiments on two OpenCSR datasets show that the proposed model achieves great performance on benchmark OpenCSR datasets.
arXiv Detail & Related papers (2023-03-18T11:15:33Z) - Zero-shot Clarifying Question Generation for Conversational Search [25.514678546942754]
We propose a constrained clarifying question generation system which uses both question templates and query facets to guide the effective and precise question generation.
Experiment results show that our method outperforms existing state-of-the-art zero-shot baselines by a large margin.
arXiv Detail & Related papers (2023-01-30T04:43:02Z) - TSGP: Two-Stage Generative Prompting for Unsupervised Commonsense
Question Answering [4.965306353273393]
Unsupervised commonsense question answering requires mining effective commonsense knowledge without the rely on the labeled task data.
We propose a two-stage prompt-based unsupervised commonsense question answering framework (TSGP)
Experimental results and analysis on three different commonsense reasoning tasks, CommonsenseQA, OpenBookQA, and SocialIQA, demonstrate that TSGP significantly improves the reasoning ability of language models in unsupervised settings.
arXiv Detail & Related papers (2022-11-24T10:19:24Z) - Differentiable Open-Ended Commonsense Reasoning [80.94997942571838]
We study open-ended commonsense reasoning (OpenCSR) using as a resource only a corpus of commonsense facts written in natural language.
As an approach to OpenCSR, we propose DrFact, an efficient Differentiable model for multi-hop Reasoning over knowledge Facts.
arXiv Detail & Related papers (2020-10-24T10:07:00Z) - Knowledgeable Dialogue Reading Comprehension on Key Turns [84.1784903043884]
Multi-choice machine reading comprehension (MRC) requires models to choose the correct answer from candidate options given a passage and a question.
Our research focuses dialogue-based MRC, where the passages are multi-turn dialogues.
It suffers from two challenges, the answer selection decision is made without support of latently helpful commonsense, and the multi-turn context may hide considerable irrelevant information.
arXiv Detail & Related papers (2020-04-29T07:04:43Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.