TopNet: Learning from Neural Topic Model to Generate Long Stories

• URL: http://arxiv.org/abs/2112.07259v1
• Date: Tue, 14 Dec 2021 09:47:53 GMT
• Title: TopNet: Learning from Neural Topic Model to Generate Long Stories
• Authors: Yazheng Yang, Boyuan Pan, Deng Cai, Huan Sun
• Abstract summary: Long story generation (LSG) is one of the coveted goals in natural language processing. We propose emphTopNet to obtain high-quality skeleton words to complement the short input. Our proposed framework is highly effective in skeleton word selection and significantly outperforms state-of-the-art models in both automatic evaluation and human evaluation.
• Score: 43.5564336855688
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Long story generation (LSG) is one of the coveted goals in natural language processing. Different from most text generation tasks, LSG requires to output a long story of rich content based on a much shorter text input, and often suffers from information sparsity. In this paper, we propose \emph{TopNet} to alleviate this problem, by leveraging the recent advances in neural topic modeling to obtain high-quality skeleton words to complement the short input. In particular, instead of directly generating a story, we first learn to map the short text input to a low-dimensional topic distribution (which is pre-assigned by a topic model). Based on this latent topic distribution, we can use the reconstruction decoder of the topic model to sample a sequence of inter-related words as a skeleton for the story. Experiments on two benchmark datasets show that our proposed framework is highly effective in skeleton word selection and significantly outperforms the state-of-the-art models in both automatic evaluation and human evaluation.

Related papers

• Enhancing Short-Text Topic Modeling with LLM-Driven Context Expansion and Prefix-Tuned VAEs [25.915607750636333]
  We propose a novel approach that leverages large language models (LLMs) to extend short texts into more detailed sequences before applying topic modeling. Our method significantly improves short-text topic modeling performance, as demonstrated by extensive experiments on real-world datasets with extreme data sparsity.
  arXiv  Detail & Related papers  (2024-10-04T01:28:56Z)
• Let the Pretrained Language Models "Imagine" for Short Texts Topic Modeling [29.87929724277381]
  In short texts, co-occurrence information is minimal, which results in feature sparsity in document representation. Existing topic models (probabilistic or neural) mostly fail to mine patterns from them to generate coherent topics. We extend short text into longer sequences using existing pre-trained language models (PLMs)
  arXiv  Detail & Related papers  (2023-10-24T00:23:30Z)
• SLCNN: Sentence-Level Convolutional Neural Network for Text Classification [0.0]
  Convolutional neural network (CNN) has shown remarkable success in the task of text classification. New baseline models have been studied for text classification using CNN. Results have shown that the proposed models have better performance, particularly in the longer documents.
  arXiv  Detail & Related papers  (2023-01-27T13:16:02Z)
• Robust Preference Learning for Storytelling via Contrastive Reinforcement Learning [53.92465205531759]
  Controlled automated story generation seeks to generate natural language stories satisfying constraints from natural language critiques or preferences. We train a contrastive bi-encoder model to align stories with human critiques, building a general purpose preference model. We further fine-tune the contrastive reward model using a prompt-learning technique to increase story generation robustness.
  arXiv  Detail & Related papers  (2022-10-14T13:21:33Z)
• Topic-Aware Encoding for Extractive Summarization [15.113768658584979]
  We propose a topic-aware encoding for document summarization to deal with this issue. A neural topic model is added in the neural-based sentence-level representation learning to adequately consider the central topic information. The experimental results on three public datasets show that our model outperforms the state-of-the-art models.
  arXiv  Detail & Related papers  (2021-12-17T15:26:37Z)
• Cue Me In: Content-Inducing Approaches to Interactive Story Generation [74.09575609958743]
  We focus on the task of interactive story generation, where the user provides the model mid-level sentence abstractions. We present two content-inducing approaches to effectively incorporate this additional information. Experimental results from both automatic and human evaluations show that these methods produce more topically coherent and personalized stories.
  arXiv  Detail & Related papers  (2020-10-20T00:36:15Z)
• Multi-View Sequence-to-Sequence Models with Conversational Structure for Abstractive Dialogue Summarization [72.54873655114844]
  Text summarization is one of the most challenging and interesting problems in NLP. This work proposes a multi-view sequence-to-sequence model by first extracting conversational structures of unstructured daily chats from different views to represent conversations. Experiments on a large-scale dialogue summarization corpus demonstrated that our methods significantly outperformed previous state-of-the-art models via both automatic evaluations and human judgment.
  arXiv  Detail & Related papers  (2020-10-04T20:12:44Z)
• Topic Adaptation and Prototype Encoding for Few-Shot Visual Storytelling [81.33107307509718]
  We propose a topic adaptive storyteller to model the ability of inter-topic generalization. We also propose a prototype encoding structure to model the ability of intra-topic derivation. Experimental results show that topic adaptation and prototype encoding structure mutually bring benefit to the few-shot model.
  arXiv  Detail & Related papers  (2020-08-11T03:55:11Z)
• Neural Data-to-Text Generation via Jointly Learning the Segmentation and Correspondence [48.765579605145454]
  We propose to explicitly segment target text into fragment units and align them with their data correspondences. The resulting architecture maintains the same expressive power as neural attention models. On both E2E and WebNLG benchmarks, we show the proposed model consistently outperforms its neural attention counterparts.
  arXiv  Detail & Related papers  (2020-05-03T14:28:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.