KGR^4: Retrieval, Retrospect, Refine and Rethink for Commonsense Generation

• URL: http://arxiv.org/abs/2112.08266v1
• Date: Wed, 15 Dec 2021 17:00:11 GMT
• Title: KGR^4: Retrieval, Retrospect, Refine and Rethink for Commonsense Generation
• Authors: Xin Liu, Dayiheng Liu, Baosong Yang, Haibo Zhang, Junwei Ding, Wenqing Yao, Weihua Luo, Haiying Zhang, Jinsong Su
• Abstract summary: We propose a Knowledge-enhanced Commonsense Generation framework, termed KGR4, consisting of four stages: Retrieval, Retrospect, Refine, Rethink. KGR4 obtains 33.56 SPICE points in the official leaderboard, outperforming the previously-reported best result by 2.49 SPICE points.
• Score: 36.78998964614422
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Generative commonsense reasoning requires machines to generate sentences describing an everyday scenario given several concepts, which has attracted much attention recently. However, existing models cannot perform as well as humans, since sentences they produce are often implausible and grammatically incorrect. In this paper, inspired by the process of humans creating sentences, we propose a novel Knowledge-enhanced Commonsense Generation framework, termed KGR^4, consisting of four stages: Retrieval, Retrospect, Refine, Rethink. Under this framework, we first perform retrieval to search for relevant sentences from external corpus as the prototypes. Then, we train the generator that either edits or copies these prototypes to generate candidate sentences, of which potential errors will be fixed by an autoencoder-based refiner. Finally, we select the output sentence from candidate sentences produced by generators with different hyper-parameters. Experimental results and in-depth analysis on the CommonGen benchmark strongly demonstrate the effectiveness of our framework. Particularly, KGR^4 obtains 33.56 SPICE points in the official leaderboard, outperforming the previously-reported best result by 2.49 SPICE points and achieving state-of-the-art performance.

Related papers

• BOOST: Harnessing Black-Box Control to Boost Commonsense in LMs' Generation [60.77990074569754]
  We present a computation-efficient framework that steers a frozen Pre-Trained Language Model towards more commonsensical generation. Specifically, we first construct a reference-free evaluator that assigns a sentence with a commonsensical score. We then use the scorer as the oracle for commonsense knowledge, and extend the controllable generation method called NADO to train an auxiliary head.
  arXiv  Detail & Related papers  (2023-10-25T23:32:12Z)
• MGR: Multi-generator Based Rationalization [14.745836934156427]
  Rationalization is to employ a generator and a predictor to construct a self-explaining NLP model. In this paper, we propose a simple yet effective method named MGR to simultaneously solve the two problems. We show that MGR improves the F1 score by up to 20.9% as compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-05-08T06:36:46Z)
• Joint Generator-Ranker Learning for Natural Language Generation [99.16268050116717]
  JGR is a novel joint training algorithm that integrates the generator and the ranker in a single framework. By iteratively updating the generator and the ranker, JGR can effectively harmonize their learning and enhance their quality jointly.
  arXiv  Detail & Related papers  (2022-06-28T12:58:30Z)
• Quark: Controllable Text Generation with Reinforced Unlearning [68.07749519374089]
  Large-scale language models often learn behaviors that are misaligned with user expectations. We introduce Quantized Reward Konditioning (Quark), an algorithm for optimizing a reward function that quantifies an (un)wanted property. For unlearning toxicity, negative sentiment, and repetition, our experiments show that Quark outperforms both strong baselines and state-of-the-art reinforcement learning methods.
  arXiv  Detail & Related papers  (2022-05-26T21:11:51Z)
• Towards Document-Level Paraphrase Generation with Sentence Rewriting and Reordering [88.08581016329398]
  We propose CoRPG (Coherence Relationship guided Paraphrase Generation) for document-level paraphrase generation. We use graph GRU to encode the coherence relationship graph and get the coherence-aware representation for each sentence. Our model can generate document paraphrase with more diversity and semantic preservation.
  arXiv  Detail & Related papers  (2021-09-15T05:53:40Z)
• Show Me How To Revise: Improving Lexically Constrained Sentence Generation with XLNet [27.567493727582736]
  We propose a two-step approach, "Predict and Revise", for constrained sentence generation. During the predict step, we leveraged the classifier to compute the learned prior for the candidate sentence. During the revise step, we resorted to MCMC sampling to revise the candidate sentence by conducting a sampled action at a sampled position drawn from the learned prior. Experimental results have demonstrated that our proposed model performs much better than the previous work in terms of sentence fluency and diversity.
  arXiv  Detail & Related papers  (2021-09-13T09:21:07Z)
• Retrieval Enhanced Model for Commonsense Generation [27.808363395849536]
  We propose a novel framework using retrieval methods to enhance both the pre-training and fine-tuning for commonsense generation. We retrieve prototype sentence candidates by concept matching and use them as auxiliary input. We demonstrate experimentally on the large-scale CommonGen benchmark that our approach achieves new state-of-the-art results.
  arXiv  Detail & Related papers  (2021-05-24T09:49:17Z)
• Commonsense Evidence Generation and Injection in Reading Comprehension [57.31927095547153]
  We propose a Commonsense Evidence Generation and Injection framework in reading comprehension, named CEGI. The framework injects two kinds of auxiliary commonsense evidence into comprehensive reading to equip the machine with the ability of rational thinking. Experiments on the CosmosQA dataset demonstrate that the proposed CEGI model outperforms the current state-of-the-art approaches.
  arXiv  Detail & Related papers  (2020-05-11T16:31:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.