Mining Logical Event Schemas From Pre-Trained Language Models

• URL: http://arxiv.org/abs/2204.05939v1
• Date: Tue, 12 Apr 2022 16:41:18 GMT
• Title: Mining Logical Event Schemas From Pre-Trained Language Models
• Authors: Lane Lawley and Lenhart Schubert
• Abstract summary: We present NESL (the Neuro-Episodic Learner), an event schema learning system that combines large language models, FrameNet parsing, and a set of simple behavioral schemas. We show that careful sampling from the language model can help emphasize stereotypical properties of situations and de-emphasize irrelevant details.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We present NESL (the Neuro-Episodic Schema Learner), an event schema learning system that combines large language models, FrameNet parsing, a powerful logical representation of language, and a set of simple behavioral schemas meant to bootstrap the learning process. In lieu of a pre-made corpus of stories, our dataset is a continuous feed of "situation samples" from a pre-trained language model, which are then parsed into FrameNet frames, mapped into simple behavioral schemas, and combined and generalized into complex, hierarchical schemas for a variety of everyday scenarios. We show that careful sampling from the language model can help emphasize stereotypical properties of situations and de-emphasize irrelevant details, and that the resulting schemas specify situations more comprehensively than those learned by other systems.

Related papers

• Matchmaker: Self-Improving Large Language Model Programs for Schema Matching [60.23571456538149]
  We propose a compositional language model program for schema matching, comprised of candidate generation, refinement and confidence scoring. Matchmaker self-improves in a zero-shot manner without the need for labeled demonstrations. Empirically, we demonstrate on real-world medical schema matching benchmarks that Matchmaker outperforms previous ML-based approaches.
  arXiv  Detail & Related papers  (2024-10-31T16:34:03Z)
• LangSuitE: Planning, Controlling and Interacting with Large Language Models in Embodied Text Environments [70.91258869156353]
  We introduce LangSuitE, a versatile and simulation-free testbed featuring 6 representative embodied tasks in textual embodied worlds. Compared with previous LLM-based testbeds, LangSuitE offers adaptability to diverse environments without multiple simulation engines. We devise a novel chain-of-thought (CoT) schema, EmMem, which summarizes embodied states w.r.t. history information.
  arXiv  Detail & Related papers  (2024-06-24T03:36:29Z)
• Drafting Event Schemas using Language Models [48.81285141287434]
  We look at the process of creating such schemas to describe complex events. Our focus is on whether we can achieve sufficient diversity and recall of key events. We show that large language models are able to achieve moderate recall against schemas taken from two different datasets.
  arXiv  Detail & Related papers  (2023-05-24T07:57:04Z)
• Autoregressive Structured Prediction with Language Models [73.11519625765301]
  We describe an approach to model structures as sequences of actions in an autoregressive manner with PLMs. Our approach achieves the new state-of-the-art on all the structured prediction tasks we looked at.
  arXiv  Detail & Related papers  (2022-10-26T13:27:26Z)
• Language Model Cascades [72.18809575261498]
  Repeated interactions at test-time with a single model, or the composition of multiple models together, further expands capabilities. Cases with control flow and dynamic structure require techniques from probabilistic programming. We formalize several existing techniques from this perspective, including scratchpads / chain of thought, verifiers, STaR, selection-inference, and tool use.
  arXiv  Detail & Related papers  (2022-07-21T07:35:18Z)
• Hidden Schema Networks [3.4123736336071864]
  We introduce a novel neural language model that enforces, via inductive biases, explicit relational structures. The model encodes sentences into sequences of symbols, which correspond to nodes visited by biased random walkers. We show that the model is able to uncover ground-truth graphs from artificially generated datasets of random token sequences.
  arXiv  Detail & Related papers  (2022-07-08T09:26:19Z)
• Show, Don't Tell: Demonstrations Outperform Descriptions for Schema-Guided Task-Oriented Dialogue [27.43338545216015]
  Show, Don't Tell is a prompt format for seq2seq modeling which uses a short labeled example dialogue to show the semantics of schema elements. While requiring similar effort from service developers, we show that using short examples as schema representations with large language models results in stronger performance and better generalization.
  arXiv  Detail & Related papers  (2022-04-08T23:27:18Z)
• Systematic Generalization on gSCAN with Language Conditioned Embedding [19.39687991647301]
  Systematic Generalization refers to a learning algorithm's ability to extrapolate learned behavior to unseen situations. We propose a novel method that learns objects' contextualized embeddings with dynamic message passing conditioned on the input natural language.
  arXiv  Detail & Related papers  (2020-09-11T17:35:05Z)
• STRIPS Action Discovery [67.73368413278631]
  Recent approaches have shown the success of classical planning at synthesizing action models even when all intermediate states are missing. We propose a new algorithm to unsupervisedly synthesize STRIPS action models with a classical planner when action signatures are unknown.
  arXiv  Detail & Related papers  (2020-01-30T17:08:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.