proScript: Partially Ordered Scripts Generation via Pre-trained Language Models

• URL: http://arxiv.org/abs/2104.08251v1
• Date: Fri, 16 Apr 2021 17:35:10 GMT
• Title: proScript: Partially Ordered Scripts Generation via Pre-trained Language Models
• Authors: Keisuke Sakaguchi, Chandra Bhagavatula, Ronan Le Bras, Niket Tandon, Peter Clark, Yejin Choi
• Abstract summary: We demonstrate for the first time that pre-trained neural language models (LMs) can be finetuned to generate high-quality scripts. We collected a large (6.4k), crowdsourced partially ordered scripts (named proScript) Our experiments show that our models perform well (e.g., F1=75.7 in task (i)), illustrating a new approach to overcoming previous barriers to script collection.
• Score: 49.03193243699244
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Scripts - standardized event sequences describing typical everyday activities - have been shown to help understand narratives by providing expectations, resolving ambiguity, and filling in unstated information. However, to date they have proved hard to author or extract from text. In this work, we demonstrate for the first time that pre-trained neural language models (LMs) can be be finetuned to generate high-quality scripts, at varying levels of granularity, for a wide range of everyday scenarios (e.g., bake a cake). To do this, we collected a large (6.4k), crowdsourced partially ordered scripts (named proScript), which is substantially larger than prior datasets, and developed models that generate scripts with combining language generation and structure prediction. We define two complementary tasks: (i) edge prediction: given a scenario and unordered events, organize the events into a valid (possibly partial-order) script, and (ii) script generation: given only a scenario, generate events and organize them into a (possibly partial-order) script. Our experiments show that our models perform well (e.g., F1=75.7 in task (i)), illustrating a new approach to overcoming previous barriers to script collection. We also show that there is still significant room for improvement toward human level performance. Together, our tasks, dataset, and models offer a new research direction for learning script knowledge.

Related papers

• MULTISCRIPT: Multimodal Script Learning for Supporting Open Domain Everyday Tasks [28.27986773292919]
  We present a new benchmark challenge -- MultiScript. For both tasks, the input consists of a target task name and a video illustrating what has been done to complete the target task. The expected output is (1) a sequence of structured step descriptions in text based on the demonstration video, and (2) a single text description for the subsequent step.
  arXiv  Detail & Related papers  (2023-10-08T01:51:17Z)
• Language Model Pre-Training with Sparse Latent Typing [66.75786739499604]
  We propose a new pre-training objective, Sparse Latent Typing, which enables the model to sparsely extract sentence-level keywords with diverse latent types. Experimental results show that our model is able to learn interpretable latent type categories in a self-supervised manner without using any external knowledge.
  arXiv  Detail & Related papers  (2022-10-23T00:37:08Z)
• Probing Script Knowledge from Pre-Trained Models [24.80244106746926]
  We design three probing tasks: inclusive sub-event selection, starting sub-event selection and temporal ordering. The three probing tasks can be further used to automatically induce a script for each main event given all the possible sub-events. Taking BERT as a case study, we conclude that the stereotypical temporal knowledge among the sub-events is well captured in BERT.
  arXiv  Detail & Related papers  (2022-04-16T05:13:39Z)
• A Generative Language Model for Few-shot Aspect-Based Sentiment Analysis [90.24921443175514]
  We focus on aspect-based sentiment analysis, which involves extracting aspect term, category, and predicting their corresponding polarities. We propose to reformulate the extraction and prediction tasks into the sequence generation task, using a generative language model with unidirectional attention. Our approach outperforms the previous state-of-the-art (based on BERT) on average performance by a large margins in few-shot and full-shot settings.
  arXiv  Detail & Related papers  (2022-04-11T18:31:53Z)
• What do Large Language Models Learn about Scripts? [5.429894958215681]
  We introduce the task of generating full event sequence descriptions given a scenario in the form of natural language prompts. In zero-shot probing experiments, we find that generative LMs produce poor ESDs with mostly omitted, irrelevant, repeated or misordered events. We propose a pipeline-based script induction framework (SIF) which can generate good quality ESDs for unseen scenarios.
  arXiv  Detail & Related papers  (2021-12-27T18:51:18Z)
• Integrating Deep Event-Level and Script-Level Information for Script Event Prediction [60.67635412135681]
  We propose a Transformer-based model, called MCPredictor, which integrates deep event-level and script-level information for script event prediction. The experimental results on the widely-used New York Times corpus demonstrate the effectiveness and superiority of the proposed model.
  arXiv  Detail & Related papers  (2021-09-24T07:37:32Z)
• Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing [78.8500633981247]
  This paper surveys and organizes research works in a new paradigm in natural language processing, which we dub "prompt-based learning" Unlike traditional supervised learning, which trains a model to take in an input x and predict an output y as P(y|x), prompt-based learning is based on language models that model the probability of text directly.
  arXiv  Detail & Related papers  (2021-07-28T18:09:46Z)
• Goal-Oriented Script Construction [23.6227797113877]
  We propose the Goal-Oriented Script Construction task, where a model produces a sequence of steps to accomplish a given goal. We pilot our task on the first multilingual script learning dataset supporting 18 languages collected from wikiHow.
  arXiv  Detail & Related papers  (2021-07-28T06:39:31Z)
• Pre-training via Paraphrasing [96.79972492585112]
  We introduce MARGE, a pre-trained sequence-to-sequence model learned with an unsupervised multi-lingual paraphrasing objective. We show it is possible to jointly learn to do retrieval and reconstruction, given only a random initialization. For example, with no additional task-specific training we achieve BLEU scores of up to 35.8 for document translation.
  arXiv  Detail & Related papers  (2020-06-26T14:43:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.