Event Detection with a Context-Aware Encoder and LoRA for Improved Performance on Long-Tailed Classes

• URL: http://arxiv.org/abs/2601.11932v1
• Date: Sat, 17 Jan 2026 06:40:55 GMT
• Title: Event Detection with a Context-Aware Encoder and LoRA for Improved Performance on Long-Tailed Classes
• Abstract summary: We investigate the architectural bottleneck for natural language understanding tasks that depend on rich, bidirectional context.<n>We focus on Macro-F1 as a more representative measure of a model's ability across the long-tail of event types.<n>Using Low-Rank Adaptation (LoRA) during finetuning provides a substantial boost in Macro-F1 scores in particular.
• Score: 5.703174155660825
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The current state of event detection research has two notable re-occurring limitations that we investigate in this study. First, the unidirectional nature of decoder-only LLMs presents a fundamental architectural bottleneck for natural language understanding tasks that depend on rich, bidirectional context. Second, we confront the conventional reliance on Micro-F1 scores in event detection literature, which systematically inflates performance by favoring majority classes. Instead, we focus on Macro-F1 as a more representative measure of a model's ability across the long-tail of event types. Our experiments demonstrate that models enhanced with sentence context achieve superior performance over canonical decoder-only baselines. Using Low-Rank Adaptation (LoRA) during finetuning provides a substantial boost in Macro-F1 scores in particular, especially for the decoder-only models, showing that LoRA can be an effective tool to enhance LLMs' performance on long-tailed event classes.

Related papers

• The Regularizing Power of Language-Training Deepfake Detectors [57.529452351298495]
  Deepfake detection generally suffers from overfitting to low-level domain-specific artifacts.<n>We use a dual-encoder architecture, pairing a frozen specialist detector with a LoRA-tuned MLLM encoder.<n>We employ a reinforcement learning stage that encourages the model to generate descriptive reasoning before classifying.
  arXiv  Detail & Related papers  (2026-05-29T12:01:17Z)
• Language Ranker: A Lightweight Ranking framework for LLM Decoding [70.01564145836129]
  This paper conceptualizes the decoding process as analogous to the ranking stage in recommendation pipelines.<n>Motivated by this insight, we propose Language Ranker, a novel framework that introduces a lightweight module to rerank candidate responses.<n> Experiments show that Language Ranker achieves performance comparable to large-scale reward models, while requiring only 0.5M additional parameters.
  arXiv  Detail & Related papers  (2025-10-23T17:56:46Z)
• L1RA: Dynamic Rank Assignment in LoRA Fine-Tuning [0.09799637101641147]
  We introduce L1RA, a technique for dynamically distributing the rank of low-rank adapters during fine-tuning using LoRA.<n>We empirically demonstrate that L1RA maintains comparable or even reduced computational overhead compared to other LoRA variants.
  arXiv  Detail & Related papers  (2025-09-05T08:03:01Z)
• GDLLM: A Global Distance-aware Modeling Approach Based on Large Language Models for Event Temporal Relation Extraction [4.6155886966345285]
  We propose a Global Distance-aware modeling approach based on Large Language Models.<n>We first present a distance-aware graph structure utilizing Graph Attention Network(GAT) to assist the LLMs in capturing long-distance dependency features.<n>We design a temporal feature learning paradigm based on soft inference to augment the identification of relations with a short-distance proximity band.
  arXiv  Detail & Related papers  (2025-08-28T14:23:39Z)
• SeFAR: Semi-supervised Fine-grained Action Recognition with Temporal Perturbation and Learning Stabilization [2.1682783789464968]
  Fine-grained Action Recognition (FAR) focuses on detailed semantic labels within shorter temporal duration.<n>Given the high costs of annotating labels and the substantial data needed for fine-tuning LLMs, we propose to adopt semi-supervised learning (SSL)<n>Our framework, SeFAR, incorporates several innovative designs to tackle these challenges.
  arXiv  Detail & Related papers  (2025-01-02T13:12:12Z)
• Unlocking Tuning-Free Few-Shot Adaptability in Visual Foundation Models by Recycling Pre-Tuned LoRAs [76.40876036912537]
  Large Language Models (LLMs) demonstrate strong few-shot adaptability without requiring fine-tuning.<n>Current Visual Foundation Models (VFMs) require explicit fine-tuning with sufficient tuning data.<n>We propose a framework, LoRA Recycle, that distills a meta-LoRA from diverse pre-tuned LoRAs with a meta-learning objective.
  arXiv  Detail & Related papers  (2024-12-03T07:25:30Z)
• Lifelong Personalized Low-Rank Adaptation of Large Language Models for Recommendation [50.837277466987345]
  We focus on the field of large language models (LLMs) for recommendation. We propose RecLoRA, which incorporates a Personalized LoRA module that maintains independent LoRAs for different users. We also design a Few2Many Learning Strategy, using a conventional recommendation model as a lens to magnify small training spaces to full spaces.
  arXiv  Detail & Related papers  (2024-08-07T04:20:28Z)
• RAR: Retrieving And Ranking Augmented MLLMs for Visual Recognition [78.97487780589574]
  Multimodal Large Language Models (MLLMs) excel at classifying fine-grained categories. This paper introduces a Retrieving And Ranking augmented method for MLLMs. Our proposed approach not only addresses the inherent limitations in fine-grained recognition but also preserves the model's comprehensive knowledge base.
  arXiv  Detail & Related papers  (2024-03-20T17:59:55Z)
• EventRL: Enhancing Event Extraction with Outcome Supervision for Large Language Models [48.136950450053476]
  EventRL is a reinforcement learning approach developed to enhance event extraction for large language models (LLMs) We evaluate EventRL against existing methods like Few-Shot Prompting (FSP) and Supervised Fine-Tuning (SFT) Our findings show that EventRL significantly outperforms these conventional approaches by improving the performance in identifying and structuring events.
  arXiv  Detail & Related papers  (2024-02-18T02:41:06Z)
• An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning [70.48605869773814]
  Catastrophic forgetting (CF) is a phenomenon that occurs in machine learning when a model forgets previously learned information.<n>This study empirically evaluates the forgetting phenomenon in large language models during continual instruction tuning.
  arXiv  Detail & Related papers  (2023-08-17T02:53:23Z)
• MA-FSAR: Multimodal Adaptation of CLIP for Few-Shot Action Recognition [41.78245303513613]
  We introduce MA-FSAR, a framework that employs the Fine-Tuning (PEFT) technique to enhance the CLIP visual encoder in terms of action-related temporal and semantic representations. In addition to these token-level designs, we propose a prototype-level text-guided construction module to further enrich the temporal and semantic characteristics of video prototypes.
  arXiv  Detail & Related papers  (2023-08-03T04:17:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.