Unsupervised Commonsense Question Answering with Self-Talk

• URL: http://arxiv.org/abs/2004.05483v2
• Date: Tue, 15 Sep 2020 18:55:05 GMT
• Title: Unsupervised Commonsense Question Answering with Self-Talk
• Authors: Vered Shwartz, Peter West, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi
• Abstract summary: We propose an unsupervised framework based on self-talk as a novel alternative to commonsense tasks. Inspired by inquiry-based discovery learning, our approach inquires language models with a number of information seeking questions. Empirical results demonstrate that the self-talk procedure substantially improves the performance of zero-shot language model baselines.
• Score: 71.63983121558843
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Natural language understanding involves reading between the lines with implicit background knowledge. Current systems either rely on pre-trained language models as the sole implicit source of world knowledge, or resort to external knowledge bases (KBs) to incorporate additional relevant knowledge. We propose an unsupervised framework based on self-talk as a novel alternative to multiple-choice commonsense tasks. Inspired by inquiry-based discovery learning (Bruner, 1961), our approach inquires language models with a number of information seeking questions such as "$\textit{what is the definition of ...}$" to discover additional background knowledge. Empirical results demonstrate that the self-talk procedure substantially improves the performance of zero-shot language model baselines on four out of six commonsense benchmarks, and competes with models that obtain knowledge from external KBs. While our approach improves performance on several benchmarks, the self-talk induced knowledge even when leading to correct answers is not always seen as useful by human judges, raising interesting questions about the inner-workings of pre-trained language models for commonsense reasoning.

Related papers

• A Knowledge Plug-and-Play Test Bed for Open-domain Dialogue Generation [51.31429493814664]
  We present a benchmark named multi-source Wizard of Wikipedia for evaluating multi-source dialogue knowledge selection and response generation. We propose a new challenge, dialogue knowledge plug-and-play, which aims to test an already trained dialogue model on using new support knowledge from previously unseen sources.
  arXiv  Detail & Related papers  (2024-03-06T06:54:02Z)
• Commonsense Knowledge Transfer for Pre-trained Language Models [83.01121484432801]
  We introduce commonsense knowledge transfer, a framework to transfer the commonsense knowledge stored in a neural commonsense knowledge model to a general-purpose pre-trained language model. It first exploits general texts to form queries for extracting commonsense knowledge from the neural commonsense knowledge model. It then refines the language model with two self-supervised objectives: commonsense mask infilling and commonsense relation prediction.
  arXiv  Detail & Related papers  (2023-06-04T15:44:51Z)
• Zero-shot Commonsense Question Answering with Cloze Translation and Consistency Optimization [20.14487209460865]
  We investigate four translation methods that can translate natural questions into cloze-style sentences. We show that our methods are complementary datasets to a knowledge base improved model, and combining them can lead to state-of-the-art zero-shot performance.
  arXiv  Detail & Related papers  (2022-01-01T07:12:49Z)
• Generated Knowledge Prompting for Commonsense Reasoning [53.88983683513114]
  We propose generating knowledge statements directly from a language model with a generic prompt format. This approach improves performance of both off-the-shelf and finetuned language models on four commonsense reasoning tasks. Notably, we find that a model's predictions can improve when using its own generated knowledge.
  arXiv  Detail & Related papers  (2021-10-15T21:58:03Z)
• Natural Language Processing with Commonsense Knowledge: A Survey [9.634283896785611]
  This paper explores the integration of commonsense knowledge into various NLP tasks. We highlight key methodologies for incorporating commonsense knowledge and their applications across different NLP tasks. The paper also examines the challenges and emerging trends in enhancing NLP systems with commonsense reasoning.
  arXiv  Detail & Related papers  (2021-08-10T13:25:29Z)
• Contextualized Knowledge-aware Attentive Neural Network: Enhancing Answer Selection with Knowledge [77.77684299758494]
  We extensively investigate approaches to enhancing the answer selection model with external knowledge from knowledge graph (KG) First, we present a context-knowledge interaction learning framework, Knowledge-aware Neural Network (KNN), which learns the QA sentence representations by considering a tight interaction with the external knowledge from KG and the textual information. To handle the diversity and complexity of KG information, we propose a Contextualized Knowledge-aware Attentive Neural Network (CKANN), which improves the knowledge representation learning with structure information via a customized Graph Convolutional Network (GCN) and comprehensively learns context-based and knowledge-based sentence representation via
  arXiv  Detail & Related papers  (2021-04-12T05:52:20Z)
• Are Pre-trained Language Models Knowledgeable to Ground Open Domain Dialogues? [20.598241369838668]
  We study knowledge-grounded dialogue generation with pre-trained language models. We find that by fine-tuning with a few dialogues containing knowledge, the pre-trained language models can outperform the state-of-the-art model.
  arXiv  Detail & Related papers  (2020-11-19T08:22:49Z)
• Knowledge-driven Data Construction for Zero-shot Evaluation in Commonsense Question Answering [80.60605604261416]
  We propose a novel neuro-symbolic framework for zero-shot question answering across commonsense tasks. We vary the set of language models, training regimes, knowledge sources, and data generation strategies, and measure their impact across tasks. We show that, while an individual knowledge graph is better suited for specific tasks, a global knowledge graph brings consistent gains across different tasks.
  arXiv  Detail & Related papers  (2020-11-07T22:52:21Z)
• Question Answering over Knowledge Base using Language Model Embeddings [0.0]
  This paper focuses on using a pre-trained language model for the Knowledge Base Question Answering task. We further fine-tuned these embeddings with a two-way attention mechanism from the knowledge base to the asked question. Our method is based on a simple Convolutional Neural Network architecture with a Multi-Head Attention mechanism to represent the asked question.
  arXiv  Detail & Related papers  (2020-10-17T22:59:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.