Probing Script Knowledge from Pre-Trained Models

• URL: http://arxiv.org/abs/2204.10176v1
• Date: Sat, 16 Apr 2022 05:13:39 GMT
• Title: Probing Script Knowledge from Pre-Trained Models
• Authors: Zijian Jin, Xingyu Zhang, Mo Yu, Lifu Huang
• Abstract summary: We design three probing tasks: inclusive sub-event selection, starting sub-event selection and temporal ordering. The three probing tasks can be further used to automatically induce a script for each main event given all the possible sub-events. Taking BERT as a case study, we conclude that the stereotypical temporal knowledge among the sub-events is well captured in BERT.
• Score: 24.80244106746926
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Script knowledge is critical for humans to understand the broad daily tasks and routine activities in the world. Recently researchers have explored the large-scale pre-trained language models (PLMs) to perform various script related tasks, such as story generation, temporal ordering of event, future event prediction and so on. However, it's still not well studied in terms of how well the PLMs capture the script knowledge. To answer this question, we design three probing tasks: inclusive sub-event selection, starting sub-event selection and temporal ordering to investigate the capabilities of PLMs with and without fine-tuning. The three probing tasks can be further used to automatically induce a script for each main event given all the possible sub-events. Taking BERT as a case study, by analyzing its performance on script induction as well as each individual probing task, we conclude that the stereotypical temporal knowledge among the sub-events is well captured in BERT, however the inclusive or starting sub-event knowledge is barely encoded.

Related papers

• Grounding Partially-Defined Events in Multimodal Data [61.0063273919745]
  We introduce a multimodal formulation for partially-defined events and cast the extraction of these events as a three-stage span retrieval task. We propose a benchmark for this task, MultiVENT-G, that consists of 14.5 hours of densely annotated current event videos and 1,168 text documents, containing 22.8K labeled event-centric entities. Results illustrate the challenges that abstract event understanding poses and demonstrates promise in event-centric video-language systems.
  arXiv  Detail & Related papers  (2024-10-07T17:59:48Z)
• TemPrompt: Multi-Task Prompt Learning for Temporal Relation Extraction in RAG-based Crowdsourcing Systems [21.312052922118585]
  Temporal relation extraction (TRE) aims to grasp the evolution of events or actions, and thus shape the workflow of associated tasks. We propose a multi-task prompt learning framework for TRE (TemPrompt), incorporating prompt tuning and contrastive learning to tackle these issues.
  arXiv  Detail & Related papers  (2024-06-21T01:52:37Z)
• TransPrompt v2: A Transferable Prompting Framework for Cross-task Text Classification [37.824031151922604]
  We propose TransPrompt v2, a novel transferable prompting framework for few-shot learning across similar or distant text classification tasks. For learning across similar tasks, we employ a multi-task meta-knowledge acquisition (MMA) procedure to train a meta-learner. For learning across distant tasks, we inject the task type descriptions into the prompt, and capture the intra-type and inter-type prompt embeddings.
  arXiv  Detail & Related papers  (2023-08-29T04:16:57Z)
• Prompt2Gaussia: Uncertain Prompt-learning for Script Event Prediction [11.54608099442562]
  Script Event Prediction (SEP) aims to predict the subsequent event for a given event chain from a candidate list. We consider public pre-trained language models as knowledge bases and automatically mine the script-related knowledge via prompt-learning. Our method, which benefits from knowledge evoked from pre-trained language models, outperforms prior baselines by 1.46% and 1.05% on two benchmarks.
  arXiv  Detail & Related papers  (2023-08-04T01:34:46Z)
• Pre-training Multi-task Contrastive Learning Models for Scientific Literature Understanding [52.723297744257536]
  Pre-trained language models (LMs) have shown effectiveness in scientific literature understanding tasks. We propose a multi-task contrastive learning framework, SciMult, to facilitate common knowledge sharing across different literature understanding tasks.
  arXiv  Detail & Related papers  (2023-05-23T16:47:22Z)
• Integrating Deep Event-Level and Script-Level Information for Script Event Prediction [60.67635412135681]
  We propose a Transformer-based model, called MCPredictor, which integrates deep event-level and script-level information for script event prediction. The experimental results on the widely-used New York Times corpus demonstrate the effectiveness and superiority of the proposed model.
  arXiv  Detail & Related papers  (2021-09-24T07:37:32Z)
• CINS: Comprehensive Instruction for Few-shot Learning in Task-oriented Dialog Systems [56.302581679816775]
  This paper proposes Comprehensive Instruction (CINS) that exploits PLMs with task-specific instructions. We design a schema (definition, constraint, prompt) of instructions and their customized realizations for three important downstream tasks in ToD. Experiments are conducted on these ToD tasks in realistic few-shot learning scenarios with small validation data.
  arXiv  Detail & Related papers  (2021-09-10T03:23:06Z)
• proScript: Partially Ordered Scripts Generation via Pre-trained Language Models [49.03193243699244]
  We demonstrate for the first time that pre-trained neural language models (LMs) can be finetuned to generate high-quality scripts. We collected a large (6.4k), crowdsourced partially ordered scripts (named proScript) Our experiments show that our models perform well (e.g., F1=75.7 in task (i)), illustrating a new approach to overcoming previous barriers to script collection.
  arXiv  Detail & Related papers  (2021-04-16T17:35:10Z)
• Temporally Correlated Task Scheduling for Sequence Learning [143.70523777803723]
  In many applications, a sequence learning task is usually associated with multiple temporally correlated auxiliary tasks. We introduce a learnable scheduler to sequence learning, which can adaptively select auxiliary tasks for training. Our method significantly improves the performance of simultaneous machine translation and stock trend forecasting.
  arXiv  Detail & Related papers  (2020-07-10T10:28:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.