Fane at SemEval-2025 Task 10: Zero-Shot Entity Framing with Large Language Models

• URL: http://arxiv.org/abs/2504.20469v1
• Date: Tue, 29 Apr 2025 07:10:53 GMT
• Title: Fane at SemEval-2025 Task 10: Zero-Shot Entity Framing with Large Language Models
• Authors: Enfa Fane, Mihai Surdeanu, Eduardo Blanco, Steven R. Corman,
• Abstract summary: We evaluate the zero-shot capabilities of large language models (LLMs) in classifying framing roles.<n>Our findings show that a hierarchical approach of first identifying broad roles and then fine-grained roles, outperforms single-step classification.
• Score: 25.283401945003277
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Understanding how news narratives frame entities is crucial for studying media's impact on societal perceptions of events. In this paper, we evaluate the zero-shot capabilities of large language models (LLMs) in classifying framing roles. Through systematic experimentation, we assess the effects of input context, prompting strategies, and task decomposition. Our findings show that a hierarchical approach of first identifying broad roles and then fine-grained roles, outperforms single-step classification. We also demonstrate that optimal input contexts and prompts vary across task levels, highlighting the need for subtask-specific strategies. We achieve a Main Role Accuracy of 89.4% and an Exact Match Ratio of 34.5%, demonstrating the effectiveness of our approach. Our findings emphasize the importance of tailored prompt design and input context optimization for improving LLM performance in entity framing.

Related papers

• Learning Task Representations from In-Context Learning [73.72066284711462]
  Large language models (LLMs) have demonstrated remarkable proficiency in in-context learning.<n>We introduce an automated formulation for encoding task information in ICL prompts as a function of attention heads.<n>We show that our method's effectiveness stems from aligning the distribution of the last hidden state with that of an optimally performing in-context-learned model.
  arXiv  Detail & Related papers  (2025-02-08T00:16:44Z)
• Active Prompt Learning with Vision-Language Model Priors [9.173468790066956]
  We introduce a class-guided clustering that leverages the pre-trained image and text encoders of vision-language models. We propose a budget-saving selective querying based on adaptive class-wise thresholds.
  arXiv  Detail & Related papers  (2024-11-23T02:34:33Z)
• Leveraging the Power of LLMs: A Fine-Tuning Approach for High-Quality Aspect-Based Summarization [25.052557735932535]
  Large language models (LLMs) have demonstrated the potential to revolutionize diverse tasks within natural language processing. This paper explores the potential of fine-tuning LLMs for the aspect-based summarization task. We evaluate the impact of fine-tuning open-source foundation LLMs, including Llama2, Mistral, Gemma and Aya, on a publicly available domain-specific aspect based summary dataset.
  arXiv  Detail & Related papers  (2024-08-05T16:00:21Z)
• Evaluating Human Alignment and Model Faithfulness of LLM Rationale [66.75309523854476]
  We study how well large language models (LLMs) explain their generations through rationales. We show that prompting-based methods are less "faithful" than attribution-based explanations.
  arXiv  Detail & Related papers  (2024-06-28T20:06:30Z)
• Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learning [79.38140606606126]
  We propose an algorithmic framework that fine-tunes vision-language models (VLMs) with reinforcement learning (RL) Our framework provides a task description and then prompts the VLM to generate chain-of-thought (CoT) reasoning. We demonstrate that our proposed framework enhances the decision-making capabilities of VLM agents across various tasks.
  arXiv  Detail & Related papers  (2024-05-16T17:50:19Z)
• Graded Relevance Scoring of Written Essays with Dense Retrieval [4.021352247826289]
  We propose a novel approach for graded relevance scoring of written essays that employs dense retrieval encoders. We leverage Contriever, which is pre-trained with contrastive learning and demonstrated comparable performance to supervised dense retrieval models. Our method establishes a new state-of-the-art performance in the task-specific scenario, while its extension for the cross-task scenario exhibited a performance that is on par with the state-of-the-art model for that scenario.
  arXiv  Detail & Related papers  (2024-05-08T16:37:58Z)
• Debiasing Multimodal Large Language Models [61.6896704217147]
  Large Vision-Language Models (LVLMs) have become indispensable tools in computer vision and natural language processing. Our investigation reveals a noteworthy bias in the generated content, where the output is primarily influenced by the underlying Large Language Models (LLMs) prior to the input image. To rectify these biases and redirect the model's focus toward vision information, we introduce two simple, training-free strategies.
  arXiv  Detail & Related papers  (2024-03-08T12:35:07Z)
• Localizing Active Objects from Egocentric Vision with Symbolic World Knowledge [62.981429762309226]
  The ability to actively ground task instructions from an egocentric view is crucial for AI agents to accomplish tasks or assist humans virtually. We propose to improve phrase grounding models' ability on localizing the active objects by: learning the role of objects undergoing change and extracting them accurately from the instructions. We evaluate our framework on Ego4D and Epic-Kitchens datasets.
  arXiv  Detail & Related papers  (2023-10-23T16:14:05Z)
• Utilizing Contextual Clues and Role Correlations for Enhancing Document-level Event Argument Extraction [14.684710634595866]
  Document-level event argument extraction is a crucial yet challenging task within the field of information extraction. Here, we introduce a novel framework named CARLG, comprising two innovative components: the Contextual Clues Aggregation (CCA) and the Role-based Latent Information Guidance (RLIG) We then instantiate the CARLG framework into two variants based on two types of current mainstream EAE approaches. Notably, our CARLG framework introduces less than 1% new parameters yet significantly improving the performance.
  arXiv  Detail & Related papers  (2023-10-08T11:09:16Z)
• Post Hoc Explanations of Language Models Can Improve Language Models [43.2109029463221]
  We present a novel framework, Amplifying Model Performance by Leveraging In-Context Learning with Post Hoc Explanations (AMPLIFY) We leverage post hoc explanation methods which output attribution scores (explanations) capturing the influence of each of the input features on model predictions. Our framework, AMPLIFY, leads to prediction accuracy improvements of about 10-25% over a wide range of tasks.
  arXiv  Detail & Related papers  (2023-05-19T04:46:04Z)
• Learning Action-Effect Dynamics for Hypothetical Vision-Language Reasoning Task [50.72283841720014]
  We propose a novel learning strategy that can improve reasoning about the effects of actions. We demonstrate the effectiveness of our proposed approach and discuss its advantages over previous baselines in terms of performance, data efficiency, and generalization capability.
  arXiv  Detail & Related papers  (2022-12-07T05:41:58Z)
• VLCDoC: Vision-Language Contrastive Pre-Training Model for Cross-Modal Document Classification [3.7798600249187295]
  Multimodal learning from document data has achieved great success lately as it allows to pre-train semantically meaningful features as a prior into a learnable downstream task. In this paper, we approach the document classification problem by learning cross-modal representations through language and vision cues. The proposed method exploits high-level interactions and learns relevant semantic information from effective attention flows within and across modalities.
  arXiv  Detail & Related papers  (2022-05-24T12:28:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.