Entity Alignment with Noisy Annotations from Large Language Models

• URL: http://arxiv.org/abs/2405.16806v2
• Date: Tue, 28 May 2024 07:50:18 GMT
• Title: Entity Alignment with Noisy Annotations from Large Language Models
• Authors: Shengyuan Chen, Qinggang Zhang, Junnan Dong, Wen Hua, Qing Li, Xiao Huang,
• Abstract summary: We propose a unified framework, LLM4EA, to effectively leverage Large Language Models for EA. Specifically, we design a novel active learning policy to significantly reduce the annotation space. We iteratively optimize the policy based on the feedback from a base EA model.
• Score: 15.189701951003611
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Entity alignment (EA) aims to merge two knowledge graphs (KGs) by identifying equivalent entity pairs. While existing methods heavily rely on human-generated labels, it is prohibitively expensive to incorporate cross-domain experts for annotation in real-world scenarios. The advent of Large Language Models (LLMs) presents new avenues for automating EA with annotations, inspired by their comprehensive capability to process semantic information. However, it is nontrivial to directly apply LLMs for EA since the annotation space in real-world KGs is large. LLMs could also generate noisy labels that may mislead the alignment. To this end, we propose a unified framework, LLM4EA, to effectively leverage LLMs for EA. Specifically, we design a novel active learning policy to significantly reduce the annotation space by prioritizing the most valuable entities based on the entire inter-KG and intra-KG structure. Moreover, we introduce an unsupervised label refiner to continuously enhance label accuracy through in-depth probabilistic reasoning. We iteratively optimize the policy based on the feedback from a base EA model. Extensive experiments demonstrate the advantages of LLM4EA on four benchmark datasets in terms of effectiveness, robustness, and efficiency. Codes are available via https://github.com/chensyCN/llm4ea_official.

Related papers

• Zero-to-Strong Generalization: Eliciting Strong Capabilities of Large Language Models Iteratively without Gold Labels [75.77877889764073]
  Large Language Models (LLMs) have demonstrated remarkable performance through supervised fine-tuning or in-context learning using gold labels. This study explores whether solely utilizing unlabeled data can elicit strong model capabilities. We propose a new paradigm termed zero-to-strong generalization.
  arXiv  Detail & Related papers  (2024-09-19T02:59:44Z)
• TnT-LLM: Text Mining at Scale with Large Language Models [24.731544646232962]
  Large Language Models (LLMs) automate the process of end-to-end label generation and assignment with minimal human effort. We show that TnT-LLM generates more accurate and relevant label when compared against state-of-the-art baselines. We also share our practical experiences and insights on the challenges and opportunities of using LLMs for large-scale text mining in real-world applications.
  arXiv  Detail & Related papers  (2024-03-18T18:45:28Z)
• Unlocking the Power of Large Language Models for Entity Alignment [29.628079581217374]
  ChatEA is an innovative framework that incorporates large language models (LLMs) to improve EA. To address the constraints of limited input KG data, ChatEA introduces a KG-code translation module. To overcome the over-reliance on entity embedding comparisons, ChatEA implements a two-stage EA strategy.
  arXiv  Detail & Related papers  (2024-02-23T01:55:35Z)
• Two Heads Are Better Than One: Integrating Knowledge from Knowledge Graphs and Large Language Models for Entity Alignment [31.70064035432789]
  We propose a Large Language Model-enhanced Entity Alignment framework (LLMEA) LLMEA identifies candidate alignments for a given entity by considering both embedding similarities between entities across Knowledge Graphs and edit distances to a virtual equivalent entity. Experiments conducted on three public datasets reveal that LLMEA surpasses leading baseline models.
  arXiv  Detail & Related papers  (2024-01-30T12:41:04Z)
• Struc-Bench: Are Large Language Models Really Good at Generating Complex Structured Data? [49.688233418425995]
  Struc-Bench is a comprehensive benchmark featuring prominent Large Language Models (LLMs) We propose two innovative metrics, P-Score (Prompting Score) and H-Score (Heuristical Score) Our experiments show that applying our structure-aware fine-tuning to LLaMA-7B leads to substantial performance gains.
  arXiv  Detail & Related papers  (2023-09-16T11:31:58Z)
• GPT-NER: Named Entity Recognition via Large Language Models [58.609582116612934]
  GPT-NER transforms the sequence labeling task to a generation task that can be easily adapted by Language Models. We find that GPT-NER exhibits a greater ability in the low-resource and few-shot setups, when the amount of training data is extremely scarce. This demonstrates the capabilities of GPT-NER in real-world NER applications where the number of labeled examples is limited.
  arXiv  Detail & Related papers  (2023-04-20T16:17:26Z)
• Ground Truth Inference for Weakly Supervised Entity Matching [76.6732856489872]
  We propose a simple but powerful labeling model for weak supervision tasks. We then tailor the labeling model specifically to the task of entity matching. We show that our labeling model results in a 9% higher F1 score on average than the best existing method.
  arXiv  Detail & Related papers  (2022-11-13T17:57:07Z)
• ActiveEA: Active Learning for Neural Entity Alignment [31.212894129845093]
  Entity alignment (EA) aims to match equivalent entities across different Knowledge Graphs (KGs) Current mainstream methods -- neural EA models -- rely on training with seed alignment, i.e., a set of pre-aligned entity pairs. We devise a novel Active Learning (AL) framework for neural EA, aiming to create highly informative seed alignment.
  arXiv  Detail & Related papers  (2021-10-13T03:38:04Z)
• MELM: Data Augmentation with Masked Entity Language Modeling for Cross-lingual NER [73.91145686634133]
  We propose a data augmentation framework with Masked-Entity Language Modeling (MELM) MELM linearizes NER labels into sentence context, and thus the fine-tuned MELM is able to predict masked tokens by explicitly conditioning on their labels. When unlabeled target data is available and MELM can be further applied to augment pseudo-labeled target data, the performance gain reaches 5.7%.
  arXiv  Detail & Related papers  (2021-08-31T07:37:43Z)
• Interpretable and Low-Resource Entity Matching via Decoupling Feature Learning from Decision Making [22.755892575582788]
  Entity Matching aims at recognizing entity records that denote the same real-world object. We propose a novel EM framework that consists of Heterogeneous Information Fusion (HIF) and Key Attribute Tree (KAT) Induction. Our method is highly efficient and outperforms SOTA EM models in most cases.
  arXiv  Detail & Related papers  (2021-06-08T08:27:31Z)
• Exploring and Evaluating Attributes, Values, and Structures for Entity Alignment [100.19568734815732]
  Entity alignment (EA) aims at building a unified Knowledge Graph (KG) of rich content by linking the equivalent entities from various KGs. attribute triples can also provide crucial alignment signal but have not been well explored yet. We propose to utilize an attributed value encoder and partition the KG into subgraphs to model the various types of attribute triples efficiently.
  arXiv  Detail & Related papers  (2020-10-07T08:03:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.