Related papers: Drafting Event Schemas using Language Models

Drafting Event Schemas using Language Models

URL: http://arxiv.org/abs/2305.14847v1
Date: Wed, 24 May 2023 07:57:04 GMT
Title: Drafting Event Schemas using Language Models
Authors: Anisha Gunjal, Greg Durrett
Abstract summary: We look at the process of creating such schemas to describe complex events. Our focus is on whether we can achieve sufficient diversity and recall of key events. We show that large language models are able to achieve moderate recall against schemas taken from two different datasets.
Score: 48.81285141287434
License: http://creativecommons.org/licenses/by/4.0/
Abstract: Past work has studied event prediction and event language modeling, sometimes mediated through structured representations of knowledge in the form of event schemas. Such schemas can lead to explainable predictions and forecasting of unseen events given incomplete information. In this work, we look at the process of creating such schemas to describe complex events. We use large language models (LLMs) to draft schemas directly in natural language, which can be further refined by human curators as necessary. Our focus is on whether we can achieve sufficient diversity and recall of key events and whether we can produce the schemas in a sufficiently descriptive style. We show that large language models are able to achieve moderate recall against schemas taken from two different datasets, with even better results when multiple prompts and multiple samples are combined. Moreover, we show that textual entailment methods can be used for both matching schemas to instances of events as well as evaluating overlap between gold and predicted schemas. Our method paves the way for easier distillation of event knowledge from large language model into schemas.

Related papers

Matchmaker: Self-Improving Large Language Model Programs for Schema Matching [60.23571456538149]
We propose a compositional language model program for schema matching, comprised of candidate generation, refinement and confidence scoring. Matchmaker self-improves in a zero-shot manner without the need for labeled demonstrations. Empirically, we demonstrate on real-world medical schema matching benchmarks that Matchmaker outperforms previous ML-based approaches.
arXiv Detail & Related papers (2024-10-31T16:34:03Z)
Prompt-based Graph Model for Joint Liberal Event Extraction and Event Schema Induction [1.3154296174423619]
Events are essential components of speech and texts, describing the changes in the state of entities. The event extraction task aims to identify and classify events and find their participants according to event schemas. The researchers propose Liberal Event Extraction (LEE), which aims to extract events and discover event schemas simultaneously.
arXiv Detail & Related papers (2024-03-19T07:56:42Z)
Open-Domain Hierarchical Event Schema Induction by Incremental Prompting and Verification [81.17473088621209]
We treat event schemas as a form of commonsense knowledge that can be derived from large language models (LLMs) We design an incremental prompting and verification method to break down the construction of a complex event graph into three stages. Compared to directly using LLMs to generate a linearized graph, our method can generate large and complex schemas with 7.2% F1 improvement in temporal relations and 31.0% F1 improvement in hierarchical relations.
arXiv Detail & Related papers (2023-07-05T01:00:44Z)
Harvesting Event Schemas from Large Language Models [38.56772862516626]
Event schema provides a conceptual, structural and formal language to represent events and model the world event knowledge. It is challenging to automatically induce high-quality and high-coverage event schemas due to the open nature of real-world events, the diversity of event expressions, and the sparsity of event knowledge. We propose a new paradigm for event schema induction -- knowledge harvesting from large-scale pre-trained language models.
arXiv Detail & Related papers (2023-05-12T06:51:05Z)
Zero-Shot On-the-Fly Event Schema Induction [61.91468909200566]
We present a new approach in which large language models are utilized to generate source documents that allow predicting, given a high-level event definition, the specific events, arguments, and relations between them. Using our model, complete schemas on any topic can be generated on-the-fly without any manual data collection, i.e., in a zero-shot manner.
arXiv Detail & Related papers (2022-10-12T14:37:00Z)
Language Model Cascades [72.18809575261498]
Repeated interactions at test-time with a single model, or the composition of multiple models together, further expands capabilities. Cases with control flow and dynamic structure require techniques from probabilistic programming. We formalize several existing techniques from this perspective, including scratchpads / chain of thought, verifiers, STaR, selection-inference, and tool use.
arXiv Detail & Related papers (2022-07-21T07:35:18Z)
Mining Logical Event Schemas From Pre-Trained Language Models [0.0]
We present NESL (the Neuro-Episodic Learner), an event schema learning system that combines large language models, FrameNet parsing, and a set of simple behavioral schemas. We show that careful sampling from the language model can help emphasize stereotypical properties of situations and de-emphasize irrelevant details.
arXiv Detail & Related papers (2022-04-12T16:41:18Z)
Future is not One-dimensional: Graph Modeling based Complex Event Schema Induction for Event Prediction [90.75260063651763]
We introduce the concept of Temporal Complex Event: a graph-based schema representation that encompasses events, arguments, temporal connections and argument relations. We release a new schema learning corpus containing 6,399 documents accompanied with event graphs, and manually constructed gold schemas.
arXiv Detail & Related papers (2021-04-13T16:41:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.