Automatic Knowledge Augmentation for Generative Commonsense Reasoning

• URL: http://arxiv.org/abs/2111.00192v1
• Date: Sat, 30 Oct 2021 06:53:48 GMT
• Title: Automatic Knowledge Augmentation for Generative Commonsense Reasoning
• Authors: Jaehyung Seo, Chanjun Park, Sugyeong Eo, Hyeonseok Moon, Heuiseok Lim
• Abstract summary: Generative commonsense reasoning is the capability of a language model to generate a sentence with a given concept-set that is based on commonsense knowledge. We propose a data-centric method that uses automatic knowledge augmentation to extend commonsense knowledge using a machine knowledge generator.
• Score: 1.1374578778690623
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Generative commonsense reasoning is the capability of a language model to generate a sentence with a given concept-set that is based on commonsense knowledge. However, generative language models still struggle to provide outputs, and the training set does not contain patterns that are sufficient for generative commonsense reasoning. In this paper, we propose a data-centric method that uses automatic knowledge augmentation to extend commonsense knowledge using a machine knowledge generator. This method can generate semi-golden sentences that improve the generative commonsense reasoning of a language model without architecture modifications. Furthermore, this approach is a model-agnostic method and does not require human effort for data construction.

Related papers

• PHALM: Building a Knowledge Graph from Scratch by Prompting Humans and a Language Model [15.148567298728574]
  We propose PHALM, a method of building a knowledge graph from scratch. We used this method to build a Japanese event knowledge graph and trained Japanese commonsense generation models.
  arXiv  Detail & Related papers  (2023-10-11T03:39:46Z)
• Commonsense Knowledge Transfer for Pre-trained Language Models [83.01121484432801]
  We introduce commonsense knowledge transfer, a framework to transfer the commonsense knowledge stored in a neural commonsense knowledge model to a general-purpose pre-trained language model. It first exploits general texts to form queries for extracting commonsense knowledge from the neural commonsense knowledge model. It then refines the language model with two self-supervised objectives: commonsense mask infilling and commonsense relation prediction.
  arXiv  Detail & Related papers  (2023-06-04T15:44:51Z)
• Generated Knowledge Prompting for Commonsense Reasoning [53.88983683513114]
  We propose generating knowledge statements directly from a language model with a generic prompt format. This approach improves performance of both off-the-shelf and finetuned language models on four commonsense reasoning tasks. Notably, we find that a model's predictions can improve when using its own generated knowledge.
  arXiv  Detail & Related papers  (2021-10-15T21:58:03Z)
• Knowledge-Grounded Dialogue Generation with Pre-trained Language Models [74.09352261943911]
  We study knowledge-grounded dialogue generation with pre-trained language models. We propose equipping response generation defined by a pre-trained language model with a knowledge selection module.
  arXiv  Detail & Related papers  (2020-10-17T16:49:43Z)
• Language Generation with Multi-Hop Reasoning on Commonsense Knowledge Graph [124.45799297285083]
  We argue that exploiting both the structural and semantic information of the knowledge graph facilitates commonsense-aware text generation. We propose Generation with Multi-Hop Reasoning Flow (GRF) that enables pre-trained models with dynamic multi-hop reasoning on multi-relational paths extracted from the external commonsense knowledge graph.
  arXiv  Detail & Related papers  (2020-09-24T13:55:32Z)
• Leap-Of-Thought: Teaching Pre-Trained Models to Systematically Reason Over Implicit Knowledge [96.92252296244233]
  Large pre-trained language models (LMs) acquire some reasoning capacity, but this ability is difficult to control. We show that LMs can be trained to reliably perform systematic reasoning combining both implicit, pre-trained knowledge and explicit natural language statements. Our work paves a path towards open-domain systems that constantly improve by interacting with users who can instantly correct a model by adding simple natural language statements.
  arXiv  Detail & Related papers  (2020-06-11T17:02:20Z)
• Knowledge Injection into Dialogue Generation via Language Models [85.65843021510521]
  InjK is a two-stage approach to inject knowledge into a dialogue generation model. First, we train a large-scale language model and query it as textual knowledge. Second, we frame a dialogue generation model to sequentially generate textual knowledge and a corresponding response.
  arXiv  Detail & Related papers  (2020-04-30T07:31:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.