GPT-RE: In-context Learning for Relation Extraction using Large Language Models

• URL: http://arxiv.org/abs/2305.02105v3
• Date: Sat, 9 Dec 2023 02:05:05 GMT
• Title: GPT-RE: In-context Learning for Relation Extraction using Large Language Models
• Authors: Zhen Wan, Fei Cheng, Zhuoyuan Mao, Qianying Liu, Haiyue Song, Jiwei Li, Sadao Kurohashi
• Abstract summary: GPT-RE bridges the gap between large language models and fully-supervised baselines in relation extraction. We evaluate GPT-RE on four widely-used RE datasets, and observe that GPT-RE achieves improvements over existing GPT-3 baselines.
• Score: 43.968903620208444
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In spite of the potential for ground-breaking achievements offered by large language models (LLMs) (e.g., GPT-3), they still lag significantly behind fully-supervised baselines (e.g., fine-tuned BERT) in relation extraction (RE). This is due to the two major shortcomings of LLMs in RE: (1) low relevance regarding entity and relation in retrieved demonstrations for in-context learning; and (2) the strong inclination to wrongly classify NULL examples into other pre-defined labels. In this paper, we propose GPT-RE to bridge the gap between LLMs and fully-supervised baselines. GPT-RE successfully addresses the aforementioned issues by (1) incorporating task-specific entity representations in demonstration retrieval; and (2) enriching the demonstrations with gold label-induced reasoning logic. We evaluate GPT-RE on four widely-used RE datasets, and observe that GPT-RE achieves improvements over not only existing GPT-3 baselines, but also fully-supervised baselines. Specifically, GPT-RE achieves SOTA performances on the Semeval and SciERC datasets, and competitive performances on the TACRED and ACE05 datasets.

Related papers

• Relation Extraction with Fine-Tuned Large Language Models in Retrieval Augmented Generation Frameworks [0.0]
  Relation Extraction (RE) is crucial for converting unstructured data into structured formats like Knowledge Graphs (KGs) Recent studies leveraging pre-trained language models (PLMs) have shown significant success in this area. This work explores the performance of fine-tuned LLMs and their integration into the Retrieval Augmented-based (RAG) RE approach.
  arXiv  Detail & Related papers  (2024-06-20T21:27:57Z)
• GenRES: Rethinking Evaluation for Generative Relation Extraction in the Era of Large Language Models [48.56814147033251]
  We introduce GenRES for a multi-dimensional assessment in terms of the topic similarity, uniqueness, granularity, factualness, and completeness of the GRE results. With GenRES, we empirically identified that precision/recall fails to justify the performance of GRE methods. Next, we conducted a human evaluation of GRE methods that shows GenRES is consistent with human preferences for RE quality.
  arXiv  Detail & Related papers  (2024-02-16T15:01:24Z)
• Rank-without-GPT: Building GPT-Independent Listwise Rerankers on Open-Source Large Language Models [59.52207546810294]
  Listwise rerankers based on large language models (LLM) are the zero-shot state-of-art. In this work, we build for the first time effective listwise rerankers without any form of dependency on GPT. Our best list se reranker surpasses the listwise rerankers based on GPT-3.5 by 13% and achieves 97% effectiveness of the ones built on GPT-4.
  arXiv  Detail & Related papers  (2023-12-05T18:57:40Z)
• SentimentGPT: Exploiting GPT for Advanced Sentiment Analysis and its Departure from Current Machine Learning [5.177947445379688]
  This study presents a thorough examination of various Generative Pretrained Transformer (GPT) methodologies in sentiment analysis. Three primary strategies are employed: 1) prompt engineering using the advanced GPT-3.5 Turbo, 2) fine-tuning GPT models, and 3) an inventive approach to embedding classification. The research yields detailed comparative insights among these strategies and individual GPT models, revealing their unique strengths and potential limitations.
  arXiv  Detail & Related papers  (2023-07-16T05:33:35Z)
• Revisiting Relation Extraction in the era of Large Language Models [24.33660998599006]
  Relation extraction (RE) is the core NLP task of inferring semantic relationships between entities from text. Recent work has instead treated the problem as a emphsequence-to-sequence task, linearizing relations between entities as target strings to be generated conditioned on the input. Here we push the limits of this approach, using larger language models (GPT-3 and Flan-T5 large) than considered in prior work and evaluating their performance on standard RE tasks under varying levels of supervision.
  arXiv  Detail & Related papers  (2023-05-08T19:19:07Z)
• GPT-NER: Named Entity Recognition via Large Language Models [58.609582116612934]
  GPT-NER transforms the sequence labeling task to a generation task that can be easily adapted by Language Models. We find that GPT-NER exhibits a greater ability in the low-resource and few-shot setups, when the amount of training data is extremely scarce. This demonstrates the capabilities of GPT-NER in real-world NER applications where the number of labeled examples is limited.
  arXiv  Detail & Related papers  (2023-04-20T16:17:26Z)
• Shall We Pretrain Autoregressive Language Models with Retrieval? A Comprehensive Study [115.96080028033904]
  We study a scalable pre-trained retrieval-augmented LM (i.e., RETRO) compared with standard GPT and retrieval-augmented GPT. Our findings highlight the promising direction of pretraining autoregressive LMs with retrieval as future foundation models.
  arXiv  Detail & Related papers  (2023-04-13T18:04:19Z)
• News Summarization and Evaluation in the Era of GPT-3 [73.48220043216087]
  We study how GPT-3 compares against fine-tuned models trained on large summarization datasets. We show that not only do humans overwhelmingly prefer GPT-3 summaries, prompted using only a task description, but these also do not suffer from common dataset-specific issues such as poor factuality.
  arXiv  Detail & Related papers  (2022-09-26T01:04:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.