Evaluation of Pipelines for Data Integration into Knowledge Graphs

• URL: http://arxiv.org/abs/2605.22304v1
• Date: Thu, 21 May 2026 10:52:48 GMT
• Title: Evaluation of Pipelines for Data Integration into Knowledge Graphs
• Abstract summary: We propose a new benchmark KGI-Bench to evaluate integration pipelines that ingest different kinds of input data into an existing KG.<n>We evaluate pipelines by analyzing their output, with the three complementary quality metrics coverage, correctness and consistency.<n>We comparatively evaluate 12 pipelines and analyze their behavior across different input data formats and design choices.
• Score: 1.5397834466394758
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Integrating new data into knowledge graphs (KG) typically involves different tasks that are executed within workflows or pipelines There are many possible pipelines for a specific integration problem but there is not yet a general approach to evaluate the overall quality and performance of such pipelines to be able to determine the best choices. We therefore propose a new benchmark KGI-Bench to evaluate integration pipelines that ingest different kinds of input data into an existing KG. We evaluate pipelines by analyzing their output, i.e., the updated KG, with the three complementary quality metrics coverage, correctness and consistency. We also provide benchmark datasets (seed KG, overlapping input data of three formats, reference KG as a ground truth) for the movie domain. To demonstrate the applicability and usefulness of the proposed benchmark, we comparatively evaluate 12 pipelines and analyze their behavior across different input data formats and design choices.

Related papers

• Integrating Meta-Features with Knowledge Graph Embeddings for Meta-Learning [8.071711833525475]
  Two crucial meta-learning tasks are pipeline performance estimation (PPE) and dataset performance-based similarity estimation (DPSE)<n>Existing approaches primarily rely on dataset meta-features to represent datasets numerically and approximate these meta-learning tasks.<n>We propose KGmetaSP, a knowledge-graph-embeddings approach that leverages existing experiment data to capture these interactions and improve both PPE and DPSE.
  arXiv  Detail & Related papers  (2026-03-20T11:57:25Z)
• Automatic End-to-End Data Integration using Large Language Models [3.188426002064626]
  We present an automatic data integration pipeline that uses GPT-5.2 to generate all artifacts required to adapt the pipeline to specific use cases.<n>We compare the performance of this LLM-based pipeline to the performance of human-designed pipelines along three case studies.<n>Our experiments show that the LLM-based pipeline is able to produce similar results, for some tasks even better results, as the human-designed pipelines.
  arXiv  Detail & Related papers  (2026-03-11T08:56:55Z)
• KGpipe: Generation and Evaluation of Pipelines for Data Integration into Knowledge Graphs [1.5397834466394758]
  Building high-quality knowledge graphs (KGs) from diverse sources requires combining methods for information extraction, data transformation, ontology mapping, entity matching, and data fusion.<n>We present a new framework, KGpipe for defining and executing pipelines that can combine existing tools or LLM (Large Language Model) functionality.
  arXiv  Detail & Related papers  (2025-11-23T09:21:14Z)
• Can Knowledge-Graph-based Retrieval Augmented Generation Really Retrieve What You Need? [57.28763506780752]
  GraphFlow is a framework that efficiently retrieves accurate and diverse knowledge required for real-world queries from text-rich KGs.<n>It outperforms strong KG-RAG baselines, including GPT-4o, by 10% on average in hit rate and recall.<n>It also shows strong generalization to unseen KGs, demonstrating its effectiveness and robustness.
  arXiv  Detail & Related papers  (2025-10-18T17:06:49Z)
• Purifying, Labeling, and Utilizing: A High-Quality Pipeline for Small Object Detection [83.90563802153707]
  PLUSNet is a high-quality Small object detection framework.<n>It comprises three components: the Hierarchical Feature (HFP) framework for purifying upstream features, the Multiple Criteria Label Assignment (MCLA) for improving the quality of midstream training samples, and the Frequency Decoupled Head (FDHead) for more effectively exploiting information to accomplish downstream tasks.
  arXiv  Detail & Related papers  (2025-04-29T10:11:03Z)
• Distill-SynthKG: Distilling Knowledge Graph Synthesis Workflow for Improved Coverage and Efficiency [59.6772484292295]
  Knowledge graphs (KGs) generated by large language models (LLMs) are increasingly valuable for Retrieval-Augmented Generation (RAG) applications. Existing KG extraction methods rely on prompt-based approaches, which are inefficient for processing large-scale corpora. We propose SynthKG, a multi-step, document-level synthesis KG workflow based on LLMs. We also design a novel graph-based retrieval framework for RAG.
  arXiv  Detail & Related papers  (2024-10-22T00:47:54Z)
• DeepfakeBench: A Comprehensive Benchmark of Deepfake Detection [55.70982767084996]
  A critical yet frequently overlooked challenge in the field of deepfake detection is the lack of a standardized, unified, comprehensive benchmark. We present the first comprehensive benchmark for deepfake detection, called DeepfakeBench, which offers three key contributions. DeepfakeBench contains 15 state-of-the-art detection methods, 9CL datasets, a series of deepfake detection evaluation protocols and analysis tools, as well as comprehensive evaluations.
  arXiv  Detail & Related papers  (2023-07-04T01:34:41Z)
• DSKReG: Differentiable Sampling on Knowledge Graph for Recommendation with Relational GNN [59.160401038969795]
  We propose differentiable sampling on Knowledge Graph for Recommendation with GNN (DSKReG) We devise a differentiable sampling strategy, which enables the selection of relevant items to be jointly optimized with the model training procedure. The experimental results demonstrate that our model outperforms state-of-the-art KG-based recommender systems.
  arXiv  Detail & Related papers  (2021-08-26T16:19:59Z)
• Better Call the Plumber: Orchestrating Dynamic Information Extraction Pipelines [1.7688094977719888]
  Plumber is the first framework that brings together the research community's disjoint IE efforts. We study the optimization problem of choosing suitable pipelines based on input sentences. Our results demonstrate the effectiveness of Plumber in dynamically generating KG information extraction pipelines.
  arXiv  Detail & Related papers  (2021-02-22T13:14:02Z)
• Exploring and Evaluating Attributes, Values, and Structures for Entity Alignment [100.19568734815732]
  Entity alignment (EA) aims at building a unified Knowledge Graph (KG) of rich content by linking the equivalent entities from various KGs. attribute triples can also provide crucial alignment signal but have not been well explored yet. We propose to utilize an attributed value encoder and partition the KG into subgraphs to model the various types of attribute triples efficiently.
  arXiv  Detail & Related papers  (2020-10-07T08:03:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.