Heuristic-Driven Link-of-Analogy Prompting: Enhancing Large Language
Models for Document-Level Event Argument Extraction
- URL: http://arxiv.org/abs/2311.06555v2
- Date: Tue, 20 Feb 2024 03:51:40 GMT
- Title: Heuristic-Driven Link-of-Analogy Prompting: Enhancing Large Language
Models for Document-Level Event Argument Extraction
- Authors: Hanzhang Zhou, Junlang Qian, Zijian Feng, Hui Lu, Zixiao Zhu, Kezhi
Mao
- Abstract summary: We introduce the Heuristic-Driven Link-of- Analogy (HD-LoA) prompting to address the challenge of example selection.
Inspired by the analogical reasoning of human, we propose the link-of-analogy prompting, which enables LLMs to process new situations.
Experiments show that our method outperforms existing prompting methods and few-shot supervised learning methods on document-level EAE datasets.
- Score: 13.42926436351462
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In this study, we investigate in-context learning (ICL) in document-level
event argument extraction (EAE) to alleviate the dependency on large-scale
labeled data for this task. We introduce the Heuristic-Driven Link-of-Analogy
(HD-LoA) prompting to address the challenge of example selection and to develop
a prompting strategy tailored for EAE. Specifically, we hypothesize and
validate that LLMs learn task-specific heuristics from demonstrations via ICL.
Building upon this hypothesis, we introduce an explicit heuristic-driven
demonstration construction approach, which transforms the haphazard example
selection process into a methodical method that emphasizes task heuristics.
Additionally, inspired by the analogical reasoning of human, we propose the
link-of-analogy prompting, which enables LLMs to process new situations by
drawing analogies to known situations, enhancing their performance on unseen
classes beyond limited ICL examples. Experiments show that our method
outperforms existing prompting methods and few-shot supervised learning methods
on document-level EAE datasets. Additionally, the HD-LoA prompting shows
effectiveness in diverse tasks like sentiment analysis and natural language
inference, demonstrating its broad adaptability.
Related papers
- Unraveling the Mechanics of Learning-Based Demonstration Selection for In-Context Learning [43.356895599336504]
We analyze the working mechanisms of the learning-based demonstration selection methods.
We empirically identify two important factors related to similarity measurement.
We introduce two effective yet simplified exemplar selection methods catering to task-agnostic and task-specific demands.
arXiv Detail & Related papers (2024-06-14T03:34:02Z) - Decompose, Enrich, and Extract! Schema-aware Event Extraction using LLMs [45.83950260830323]
This work focuses on harnessing Large Language Models for automated Event Extraction.
It introduces a new method to address hallucination by decomposing the task into Event Detection and Event Argument Extraction.
arXiv Detail & Related papers (2024-06-03T06:55:10Z) - Prompt Optimization with EASE? Efficient Ordering-aware Automated Selection of Exemplars [66.823588073584]
Large language models (LLMs) have shown impressive capabilities in real-world applications.
The quality of these exemplars in the prompt greatly impacts performance.
Existing methods fail to adequately account for the impact of exemplar ordering on the performance.
arXiv Detail & Related papers (2024-05-25T08:23:05Z) - Inverse-RLignment: Inverse Reinforcement Learning from Demonstrations for LLM Alignment [62.05713042908654]
We introduce Alignment from Demonstrations (AfD), a novel approach leveraging high-quality demonstration data to overcome these challenges.
We formalize AfD within a sequential decision-making framework, highlighting its unique challenge of missing reward signals.
Practically, we propose a computationally efficient algorithm that extrapolates over a tailored reward model for AfD.
arXiv Detail & Related papers (2024-05-24T15:13:53Z) - DEEP-ICL: Definition-Enriched Experts for Language Model In-Context Learning [37.22553531518853]
It has long been assumed that the sheer number of parameters in large language models (LLMs) drives in-context learning (ICL) capabilities.
We introduce DEEP-ICL, a novel task Definition Enriched ExPert Ensembling methodology for ICL.
We argue that improvement from ICL does not directly rely on model size, but essentially stems from understanding task definitions and task-guided learning.
arXiv Detail & Related papers (2024-03-07T05:26:41Z) - C-ICL: Contrastive In-context Learning for Information Extraction [54.39470114243744]
c-ICL is a novel few-shot technique that leverages both correct and incorrect sample constructions to create in-context learning demonstrations.
Our experiments on various datasets indicate that c-ICL outperforms previous few-shot in-context learning methods.
arXiv Detail & Related papers (2024-02-17T11:28:08Z) - Revisiting Demonstration Selection Strategies in In-Context Learning [66.11652803887284]
Large language models (LLMs) have shown an impressive ability to perform a wide range of tasks using in-context learning (ICL)
In this work, we first revisit the factors contributing to this variance from both data and model aspects, and find that the choice of demonstration is both data- and model-dependent.
We propose a data- and model-dependent demonstration selection method, textbfTopK + ConE, based on the assumption that textitthe performance of a demonstration positively correlates with its contribution to the model's understanding of the test samples.
arXiv Detail & Related papers (2024-01-22T16:25:27Z) - Iterative Forward Tuning Boosts In-Context Learning in Language Models [88.25013390669845]
In this study, we introduce a novel two-stage framework to boost in-context learning in large language models (LLMs)
Specifically, our framework delineates the ICL process into two distinct stages: Deep-Thinking and test stages.
The Deep-Thinking stage incorporates a unique attention mechanism, i.e., iterative enhanced attention, which enables multiple rounds of information accumulation.
arXiv Detail & Related papers (2023-05-22T13:18:17Z) - Post Hoc Explanations of Language Models Can Improve Language Models [43.2109029463221]
We present a novel framework, Amplifying Model Performance by Leveraging In-Context Learning with Post Hoc Explanations (AMPLIFY)
We leverage post hoc explanation methods which output attribution scores (explanations) capturing the influence of each of the input features on model predictions.
Our framework, AMPLIFY, leads to prediction accuracy improvements of about 10-25% over a wide range of tasks.
arXiv Detail & Related papers (2023-05-19T04:46:04Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.