A Knowledge-Enhanced Pretraining Model for Commonsense Story Generation

• URL: http://arxiv.org/abs/2001.05139v1
• Date: Wed, 15 Jan 2020 05:42:27 GMT
• Title: A Knowledge-Enhanced Pretraining Model for Commonsense Story Generation
• Authors: Jian Guan, Fei Huang, Zhihao Zhao, Xiaoyan Zhu, Minlie Huang
• Abstract summary: We propose to utilize commonsense knowledge from external knowledge bases to generate reasonable stories. We employ multi-task learning which combines a discriminative objective to distinguish true and fake stories. Our model can generate more reasonable stories than state-of-the-art baselines, particularly in terms of logic and global coherence.
• Score: 98.25464306634758
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Story generation, namely generating a reasonable story from a leading context, is an important but challenging task. In spite of the success in modeling fluency and local coherence, existing neural language generation models (e.g., GPT-2) still suffer from repetition, logic conflicts, and lack of long-range coherence in generated stories. We conjecture that this is because of the difficulty of associating relevant commonsense knowledge, understanding the causal relationships, and planning entities and events with proper temporal order. In this paper, we devise a knowledge-enhanced pretraining model for commonsense story generation. We propose to utilize commonsense knowledge from external knowledge bases to generate reasonable stories. To further capture the causal and temporal dependencies between the sentences in a reasonable story, we employ multi-task learning which combines a discriminative objective to distinguish true and fake stories during fine-tuning. Automatic and manual evaluation shows that our model can generate more reasonable stories than state-of-the-art baselines, particularly in terms of logic and global coherence.

Related papers

• Commonsense Knowledge Transfer for Pre-trained Language Models [83.01121484432801]
  We introduce commonsense knowledge transfer, a framework to transfer the commonsense knowledge stored in a neural commonsense knowledge model to a general-purpose pre-trained language model. It first exploits general texts to form queries for extracting commonsense knowledge from the neural commonsense knowledge model. It then refines the language model with two self-supervised objectives: commonsense mask infilling and commonsense relation prediction.
  arXiv  Detail & Related papers  (2023-06-04T15:44:51Z)
• Neural Story Planning [8.600049807193413]
  We present an approach to story plot generation that unifies causal planning with neural language models. Our system infers the preconditions for events in the story and then events that will cause those conditions to become true. Results indicate that our proposed method produces more coherent plotlines than several strong baselines.
  arXiv  Detail & Related papers  (2022-12-16T21:29:41Z)
• Robust Preference Learning for Storytelling via Contrastive Reinforcement Learning [53.92465205531759]
  Controlled automated story generation seeks to generate natural language stories satisfying constraints from natural language critiques or preferences. We train a contrastive bi-encoder model to align stories with human critiques, building a general purpose preference model. We further fine-tune the contrastive reward model using a prompt-learning technique to increase story generation robustness.
  arXiv  Detail & Related papers  (2022-10-14T13:21:33Z)
• Incorporating Commonsense Knowledge into Story Ending Generation via Heterogeneous Graph Networks [16.360265861788253]
  We propose a Story Heterogeneous Graph Network (SHGN) to explicitly model both the information of story context at different levels and the multi-grained interactive relations among them. In detail, we consider commonsense knowledge, words and sentences as three types of nodes. We design two auxiliary tasks to implicitly capture the sentiment trend and key events lie in the context.
  arXiv  Detail & Related papers  (2022-01-29T09:33:11Z)
• Guiding Neural Story Generation with Reader Models [5.935317028008691]
  We introduce Story generation with Reader Models (StoRM), a framework in which a reader model is used to reason about the story should progress. Experiments show that our model produces significantly more coherent and on-topic stories, outperforming baselines in dimensions including plot plausibility and staying on topic.
  arXiv  Detail & Related papers  (2021-12-16T03:44:01Z)
• iReason: Multimodal Commonsense Reasoning using Videos and Natural Language with Interpretability [0.0]
  Causality knowledge is vital to building robust AI systems. We propose iReason, a framework that infers visual-semantic commonsense knowledge using both videos and natural language captions.
  arXiv  Detail & Related papers  (2021-06-25T02:56:34Z)
• Fact-driven Logical Reasoning for Machine Reading Comprehension [82.58857437343974]
  We are motivated to cover both commonsense and temporary knowledge clues hierarchically. Specifically, we propose a general formalism of knowledge units by extracting backbone constituents of the sentence. We then construct a supergraph on top of the fact units, allowing for the benefit of sentence-level (relations among fact groups) and entity-level interactions.
  arXiv  Detail & Related papers  (2021-05-21T13:11:13Z)
• Language Generation with Multi-Hop Reasoning on Commonsense Knowledge Graph [124.45799297285083]
  We argue that exploiting both the structural and semantic information of the knowledge graph facilitates commonsense-aware text generation. We propose Generation with Multi-Hop Reasoning Flow (GRF) that enables pre-trained models with dynamic multi-hop reasoning on multi-relational paths extracted from the external commonsense knowledge graph.
  arXiv  Detail & Related papers  (2020-09-24T13:55:32Z)
• Knowledge Injection into Dialogue Generation via Language Models [85.65843021510521]
  InjK is a two-stage approach to inject knowledge into a dialogue generation model. First, we train a large-scale language model and query it as textual knowledge. Second, we frame a dialogue generation model to sequentially generate textual knowledge and a corresponding response.
  arXiv  Detail & Related papers  (2020-04-30T07:31:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.