Rainier: Reinforced Knowledge Introspector for Commonsense Question Answering

• URL: http://arxiv.org/abs/2210.03078v1
• Date: Thu, 6 Oct 2022 17:34:06 GMT
• Title: Rainier: Reinforced Knowledge Introspector for Commonsense Question Answering
• Authors: Jiacheng Liu, Skyler Hallinan, Ximing Lu, Pengfei He, Sean Welleck, Hannaneh Hajishirzi, Yejin Choi
• Abstract summary: We present Rainier, or Reinforced Knowledge Introspector, that learns to generate contextually relevant knowledge in response to given questions. Our approach starts by imitating knowledge generated by GPT-3, then learns to generate its own knowledge via reinforcement learning. Our work is the first to report that knowledge generated by models that are orders of magnitude smaller than GPT-3, even without direct supervision on the knowledge itself, can exceed the quality of knowledge elicited from GPT-3 for commonsense QA.
• Score: 74.90418840431425
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Knowledge underpins reasoning. Recent research demonstrates that when relevant knowledge is provided as additional context to commonsense question answering (QA), it can substantially enhance the performance even on top of state-of-the-art. The fundamental challenge is where and how to find such knowledge that is high quality and on point with respect to the question; knowledge retrieved from knowledge bases are incomplete and knowledge generated from language models are inconsistent. We present Rainier, or Reinforced Knowledge Introspector, that learns to generate contextually relevant knowledge in response to given questions. Our approach starts by imitating knowledge generated by GPT-3, then learns to generate its own knowledge via reinforcement learning where rewards are shaped based on the increased performance on the resulting question answering. Rainier demonstrates substantial and consistent performance gains when tested over 9 different commonsense benchmarks: including 5 in-domain benchmarks that are seen during reinforcement learning, as well as 4 out-of-domain benchmarks that are kept unseen. Our work is the first to report that knowledge generated by models that are orders of magnitude smaller than GPT-3, even without direct supervision on the knowledge itself, can exceed the quality of knowledge elicited from GPT-3 for commonsense QA.

Related papers

• Knowledge Condensation and Reasoning for Knowledge-based VQA [20.808840633377343]
  Recent studies retrieve the knowledge passages from external knowledge bases and then use them to answer questions. We propose two synergistic models: Knowledge Condensation model and Knowledge Reasoning model. Our method achieves state-of-the-art performance on knowledge-based VQA datasets.
  arXiv  Detail & Related papers  (2024-03-15T06:06:06Z)
• Beyond Factuality: A Comprehensive Evaluation of Large Language Models as Knowledge Generators [78.63553017938911]
  Large language models (LLMs) outperform information retrieval techniques for downstream knowledge-intensive tasks. However, community concerns abound regarding the factuality and potential implications of using this uncensored knowledge. We introduce CONNER, designed to evaluate generated knowledge from six important perspectives.
  arXiv  Detail & Related papers  (2023-10-11T08:22:37Z)
• Self-Knowledge Guided Retrieval Augmentation for Large Language Models [59.771098292611846]
  Large language models (LLMs) have shown superior performance without task-specific fine-tuning. Retrieval-based methods can offer non-parametric world knowledge and improve the performance on tasks such as question answering. Self-Knowledge guided Retrieval augmentation (SKR) is a simple yet effective method which can let LLMs refer to the questions they have previously encountered.
  arXiv  Detail & Related papers  (2023-10-08T04:22:33Z)
• A Unified End-to-End Retriever-Reader Framework for Knowledge-based VQA [67.75989848202343]
  This paper presents a unified end-to-end retriever-reader framework towards knowledge-based VQA. We shed light on the multi-modal implicit knowledge from vision-language pre-training models to mine its potential in knowledge reasoning. Our scheme is able to not only provide guidance for knowledge retrieval, but also drop these instances potentially error-prone towards question answering.
  arXiv  Detail & Related papers  (2022-06-30T02:35:04Z)
• ArT: All-round Thinker for Unsupervised Commonsense Question-Answering [54.068032948300655]
  We propose an approach of All-round Thinker (ArT) by fully taking association during knowledge generating. We evaluate it on three commonsense QA benchmarks: COPA, SocialIQA and SCT.
  arXiv  Detail & Related papers  (2021-12-26T18:06:44Z)
• Coarse-to-Careful: Seeking Semantic-related Knowledge for Open-domain Commonsense Question Answering [12.406729445165857]
  It is prevalent to utilize external knowledge to help machine answer questions that need background commonsense. We propose a semantic-driven knowledge-aware QA framework, which controls the knowledge injection in a coarse-to-careful fashion.
  arXiv  Detail & Related papers  (2021-07-04T10:56:36Z)
• Enhancing Question Generation with Commonsense Knowledge [33.289599417096206]
  We propose a multi-task learning framework to introduce commonsense knowledge into question generation process. Experimental results on SQuAD show that our proposed methods are able to noticeably improve the QG performance on both automatic and human evaluation metrics.
  arXiv  Detail & Related papers  (2021-06-19T08:58:13Z)
• Benchmarking Knowledge-Enhanced Commonsense Question Answering via Knowledge-to-Text Transformation [30.38055266965927]
  We investigate how far can we get by exploiting external knowledge for Commonsense Question Answering. We benchmark knowledge-enhanced CQA using a simple and effective knowledge-to-text transformation framework. Experiments show that our knowledge-to-text framework is effective and state-of-the-art performance on CommonsenseQA dataset.
  arXiv  Detail & Related papers  (2021-01-04T04:29:03Z)
• KRISP: Integrating Implicit and Symbolic Knowledge for Open-Domain Knowledge-Based VQA [107.7091094498848]
  One of the most challenging question types in VQA is when answering the question requires outside knowledge not present in the image. In this work we study open-domain knowledge, the setting when the knowledge required to answer a question is not given/annotated, neither at training nor test time. We tap into two types of knowledge representations and reasoning. First, implicit knowledge which can be learned effectively from unsupervised language pre-training and supervised training data with transformer-based models.
  arXiv  Detail & Related papers  (2020-12-20T20:13:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.