RecurrentGPT: Interactive Generation of (Arbitrarily) Long Text

• URL: http://arxiv.org/abs/2305.13304v1
• Date: Mon, 22 May 2023 17:58:10 GMT
• Title: RecurrentGPT: Interactive Generation of (Arbitrarily) Long Text
• Authors: Wangchunshu Zhou, Yuchen Eleanor Jiang, Peng Cui, Tiannan Wang, Zhenxin Xiao, Yifan Hou, Ryan Cotterell, Mrinmaya Sachan
• Abstract summary: We introduce RecurrentGPT, a language-based simulacrum of the recurrence mechanism in RNNs. At each timestep, RecurrentGPT generates a paragraph of text and updates its language-based long-short term memory. RecurrentGPT is an initial step towards next-generation computer-assisted writing systems.
• Score: 81.33699837678229
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The fixed-size context of Transformer makes GPT models incapable of generating arbitrarily long text. In this paper, we introduce RecurrentGPT, a language-based simulacrum of the recurrence mechanism in RNNs. RecurrentGPT is built upon a large language model (LLM) such as ChatGPT and uses natural language to simulate the Long Short-Term Memory mechanism in an LSTM. At each timestep, RecurrentGPT generates a paragraph of text and updates its language-based long-short term memory stored on the hard drive and the prompt, respectively. This recurrence mechanism enables RecurrentGPT to generate texts of arbitrary length without forgetting. Since human users can easily observe and edit the natural language memories, RecurrentGPT is interpretable and enables interactive generation of long text. RecurrentGPT is an initial step towards next-generation computer-assisted writing systems beyond local editing suggestions. In addition to producing AI-generated content (AIGC), we also demonstrate the possibility of using RecurrentGPT as an interactive fiction that directly interacts with consumers. We call this usage of generative models by ``AI As Contents'' (AIAC), which we believe is the next form of conventional AIGC. We further demonstrate the possibility of using RecurrentGPT to create personalized interactive fiction that directly interacts with readers instead of interacting with writers. More broadly, RecurrentGPT demonstrates the utility of borrowing ideas from popular model designs in cognitive science and deep learning for prompting LLMs. Our code is available at https://github.com/aiwaves-cn/RecurrentGPT and an online demo is available at https://www.aiwaves.org/recurrentgpt.

Related papers

• Large Generative Model-assisted Talking-face Semantic Communication System [55.42631520122753]
  This study introduces a Large Generative Model-assisted Talking-face Semantic Communication (LGM-TSC) system. Generative Semantic Extractor (GSE) at the transmitter converts semantically sparse talking-face videos into texts with high information density. Private Knowledge Base (KB) based on the Large Language Model (LLM) for semantic disambiguation and correction. Generative Semantic Reconstructor (GSR) that utilizes BERT-VITS2 and SadTalker models to transform text back into a high-QoE talking-face video.
  arXiv  Detail & Related papers  (2024-11-06T12:45:46Z)
• 'Quis custodiet ipsos custodes?' Who will watch the watchmen? On Detecting AI-generated peer-reviews [20.030884734361358]
  There is a growing concern that AI-generated texts could compromise scientific publishing, including peer-review. We introduce the Term Frequency (TF) model, which posits that AI often repeats tokens, and the Review Regeneration (RR) model, which is based on the idea that ChatGPT generates similar outputs upon re-prompting. Our findings suggest both our proposed methods perform better than the other AI text detectors.
  arXiv  Detail & Related papers  (2024-10-13T08:06:08Z)
• AuthentiGPT: Detecting Machine-Generated Text via Black-Box Language Models Denoising [4.924903495092775]
  Large language models (LLMs) create text that closely mimics human writing, which can lead to potential misuse. We present AuthentiGPT, an efficient classifier that distinguishes between machine-generated and human-written texts. With a 0.918 AUROC score on a domain-specific dataset, AuthentiGPT demonstrates its effectiveness over other commercial algorithms.
  arXiv  Detail & Related papers  (2023-11-13T19:36:54Z)
• AmadeusGPT: a natural language interface for interactive animal behavioral analysis [65.55906175884748]
  We introduce AmadeusGPT: a natural language interface that turns natural language descriptions of behaviors into machine-executable code. We show we can produce state-of-the-art performance on the MABE 2022 behavior challenge tasks. AmadeusGPT presents a novel way to merge deep biological knowledge, large-language models, and core computer vision modules into a more naturally intelligent system.
  arXiv  Detail & Related papers  (2023-07-10T19:15:17Z)
• Unsupervised Text Embedding Space Generation Using Generative Adversarial Networks for Text Synthesis [0.43512163406551996]
  We propose Text Embedding Space Generative Adversarial Networks (TESGAN) to solve the gradient backpropagation problem. TESGAN conducts unsupervised learning which does not directly refer to the text of the training data to overcome the data memorization issue.
  arXiv  Detail & Related papers  (2023-06-19T10:22:12Z)
• Transformer Working Memory Enables Regular Language Reasoning and Natural Language Length Extrapolation [72.71398034617607]
  We propose a new Transformer variant named RegularGPT. With its novel combination of Weight-Sharing, Adaptive-Depth, and Sliding-Dilated-Attention, RegularGPT constructs working memory along the depth dimension. We test RegularGPT on the task of natural language length extrapolation and surprisingly find that it rediscovers the local windowed attention effect.
  arXiv  Detail & Related papers  (2023-05-05T18:54:40Z)
• Multiversal views on language models [0.0]
  We present a framework in which generative language models are conceptualized as multiverse generators. This framework also applies to human imagination and is core to how we read and write fiction. We call for exploration into this commonality through new forms of interfaces which allow humans to couple their imagination to AI to write.
  arXiv  Detail & Related papers  (2021-02-12T08:28:28Z)
• POINTER: Constrained Progressive Text Generation via Insertion-based Generative Pre-training [93.79766670391618]
  We present POINTER, a novel insertion-based approach for hard-constrained text generation. The proposed method operates by progressively inserting new tokens between existing tokens in a parallel manner. The resulting coarse-to-fine hierarchy makes the generation process intuitive and interpretable.
  arXiv  Detail & Related papers  (2020-05-01T18:11:54Z)
• PALM: Pre-training an Autoencoding&Autoregressive Language Model for Context-conditioned Generation [92.7366819044397]
  Self-supervised pre-training has emerged as a powerful technique for natural language understanding and generation. This work presents PALM with a novel scheme that jointly pre-trains an autoencoding and autoregressive language model on a large unlabeled corpus. An extensive set of experiments show that PALM achieves new state-of-the-art results on a variety of language generation benchmarks.
  arXiv  Detail & Related papers  (2020-04-14T06:25:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.