LLM with Relation Classifier for Document-Level Relation Extraction

• URL: http://arxiv.org/abs/2408.13889v2
• Date: Sat, 07 Dec 2024 09:43:20 GMT
• Title: LLM with Relation Classifier for Document-Level Relation Extraction
• Authors: Xingzuo Li, Kehai Chen, Yunfei Long, Min Zhang,
• Abstract summary: Large language models (LLMs) have created a new paradigm for natural language processing. This paper investigates the causes of this performance gap, identifying the dispersion of attention by LLMs due to entity pairs without relations as a key factor.
• Score: 25.587850398830252
• License:
• Abstract: Large language models (LLMs) have created a new paradigm for natural language processing. Despite their advancement, LLM-based methods still lag behind traditional approaches in document-level relation extraction (DocRE), a critical task for understanding complex entity relations within long context. This paper investigates the causes of this performance gap, identifying the dispersion of attention by LLMs due to entity pairs without relations as a key factor. We then introduce a novel classifier-LLM approach to DocRE. Particularly, the proposed approach begins with a classifier designed to select entity pair candidates that exhibit potential relations and then feed them to LLM for final relation classification. This method ensures that the LLM's attention is directed at relation-expressing entity pairs instead of those without relations during inference. Experiments on DocRE benchmarks reveal that our method significantly outperforms recent LLM-based DocRE models and narrows the performance gap with state-of-the-art BERT-based models.

Related papers

• LLM-Lasso: A Robust Framework for Domain-Informed Feature Selection and Regularization [59.75242204923353]
  We introduce LLM-Lasso, a framework that leverages large language models (LLMs) to guide feature selection in Lasso regression. LLMs generate penalty factors for each feature, which are converted into weights for the Lasso penalty using a simple, tunable model. Features identified as more relevant by the LLM receive lower penalties, increasing their likelihood of being retained in the final model.
  arXiv  Detail & Related papers  (2025-02-15T02:55:22Z)
• Preference Leakage: A Contamination Problem in LLM-as-a-judge [69.96778498636071]
  Large Language Models (LLMs) as judges and LLM-based data synthesis have emerged as two fundamental LLM-driven data annotation methods. In this work, we expose preference leakage, a contamination problem in LLM-as-a-judge caused by the relatedness between the synthetic data generators and LLM-based evaluators.
  arXiv  Detail & Related papers  (2025-02-03T17:13:03Z)
• Harnessing Large Language Models for Knowledge Graph Question Answering via Adaptive Multi-Aspect Retrieval-Augmentation [81.18701211912779]
  We introduce an Adaptive Multi-Aspect Retrieval-augmented over KGs (Amar) framework. This method retrieves knowledge including entities, relations, and subgraphs, and converts each piece of retrieved text into prompt embeddings. Our method has achieved state-of-the-art performance on two common datasets.
  arXiv  Detail & Related papers  (2024-12-24T16:38:04Z)
• Will LLMs Replace the Encoder-Only Models in Temporal Relation Classification? [2.1861408994125253]
  Large Language Models (LLM) have recently shown promising performance in temporal reasoning tasks. Recent studies have tested the LLMs' performance in detecting temporal relations of closed-source models only.
  arXiv  Detail & Related papers  (2024-10-14T13:10:45Z)
• Enriching Ontologies with Disjointness Axioms using Large Language Models [5.355177558868206]
  Large Models (LLMs) offer consistency by identifying and asserting class disjointness axioms. Our approach aims at leveraging the implicit knowledge embedded in LLMs to elicit knowledge for classifying ontological disjointness. Our findings suggest that LLMs, when guided by effective prompt strategies, can reliably identify disjointness relationships.
  arXiv  Detail & Related papers  (2024-10-04T09:00:06Z)
• Beyond Inter-Item Relations: Dynamic Adaption for Enhancing LLM-Based Sequential Recommendation [83.87767101732351]
  Sequential recommender systems (SRS) predict the next items that users may prefer based on user historical interaction sequences. Inspired by the rise of large language models (LLMs) in various AI applications, there is a surge of work on LLM-based SRS. We propose DARec, a sequential recommendation model built on top of coarse-grained adaption for capturing inter-item relations.
  arXiv  Detail & Related papers  (2024-08-14T10:03:40Z)
• Are LLMs Good Annotators for Discourse-level Event Relation Extraction? [15.365993658296016]
  Large Language Models (LLMs) have demonstrated proficiency in a wide array of natural language processing tasks. Our study reveals a notable underperformance of LLMs compared to the baseline established through supervised learning.
  arXiv  Detail & Related papers  (2024-07-28T19:27:06Z)
• Relation Extraction with Fine-Tuned Large Language Models in Retrieval Augmented Generation Frameworks [0.0]
  Relation Extraction (RE) is crucial for converting unstructured data into structured formats like Knowledge Graphs (KGs) Recent studies leveraging pre-trained language models (PLMs) have shown significant success in this area. This work explores the performance of fine-tuned LLMs and their integration into the Retrieval Augmented-based (RAG) RE approach.
  arXiv  Detail & Related papers  (2024-06-20T21:27:57Z)
• Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding [78.36702055076456]
  This paper introduces Multi-scale Positional. (Ms-PoE) which is a simple yet effective plug-and-play approach to enhance the capacity of. LLMs to handle relevant information located in the middle of the context.
  arXiv  Detail & Related papers  (2024-03-05T04:58:37Z)
• Semi-automatic Data Enhancement for Document-Level Relation Extraction with Distant Supervision from Large Language Models [26.523153535336725]
  Document-level Relation Extraction (DocRE) aims to extract relations from a long context. We propose a method integrating a large language model (LLM) and a natural language inference (NLI) module to generate relation triples. We demonstrate the effectiveness of our approach by introducing an enhanced dataset known as DocGNRE.
  arXiv  Detail & Related papers  (2023-11-13T13:10:44Z)
• Improving Open Information Extraction with Large Language Models: A Study on Demonstration Uncertainty [52.72790059506241]
  Open Information Extraction (OIE) task aims at extracting structured facts from unstructured text. Despite the potential of large language models (LLMs) like ChatGPT as a general task solver, they lag behind state-of-the-art (supervised) methods in OIE tasks.
  arXiv  Detail & Related papers  (2023-09-07T01:35:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.