CoDEx: A Comprehensive Knowledge Graph Completion Benchmark

• URL: http://arxiv.org/abs/2009.07810v2
• Date: Tue, 6 Oct 2020 09:10:10 GMT
• Title: CoDEx: A Comprehensive Knowledge Graph Completion Benchmark
• Authors: Tara Safavi, Danai Koutra
• Abstract summary: CoDEx is a set of knowledge graph completion datasets extracted from Wikidata and Wikipedia. CoDEx comprises three knowledge graphs varying in size and structure, multilingual descriptions of entities and relations, and tens of thousands of hard negative triples.
• Score: 16.454849794911084
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present CoDEx, a set of knowledge graph completion datasets extracted from Wikidata and Wikipedia that improve upon existing knowledge graph completion benchmarks in scope and level of difficulty. In terms of scope, CoDEx comprises three knowledge graphs varying in size and structure, multilingual descriptions of entities and relations, and tens of thousands of hard negative triples that are plausible but verified to be false. To characterize CoDEx, we contribute thorough empirical analyses and benchmarking experiments. First, we analyze each CoDEx dataset in terms of logical relation patterns. Next, we report baseline link prediction and triple classification results on CoDEx for five extensively tuned embedding models. Finally, we differentiate CoDEx from the popular FB15K-237 knowledge graph completion dataset by showing that CoDEx covers more diverse and interpretable content, and is a more difficult link prediction benchmark. Data, code, and pretrained models are available at https://bit.ly/2EPbrJs.

Related papers

• GraCoRe: Benchmarking Graph Comprehension and Complex Reasoning in Large Language Models [22.705728671135834]
  This paper presents GraCoRe, a benchmark for systematically assessing Large Language Models' graph comprehension and reasoning. GraCoRe uses a three-tier hierarchical taxonomy to categorize and test models on pure graph and heterogeneous graphs. Key findings reveal that semantic enrichment enhances reasoning performance, node ordering impacts task success, and the ability to process longer texts does not necessarily improve graph comprehension or reasoning.
  arXiv  Detail & Related papers  (2024-07-03T09:12:38Z)
• Wiki Entity Summarization Benchmark [9.25319552487389]
  Entity summarization aims to compute concise summaries for entities in knowledge graphs. Existing datasets and benchmarks are often limited to a few hundred entities. We propose WikES, a comprehensive benchmark comprising of entities, their summaries, and their connections.
  arXiv  Detail & Related papers  (2024-06-12T17:22:00Z)
• Challenging the Myth of Graph Collaborative Filtering: a Reasoned and Reproducibility-driven Analysis [50.972595036856035]
  We present a code that successfully replicates results from six popular and recent graph recommendation models. We compare these graph models with traditional collaborative filtering models that historically performed well in offline evaluations. By investigating the information flow from users' neighborhoods, we aim to identify which models are influenced by intrinsic features in the dataset structure.
  arXiv  Detail & Related papers  (2023-08-01T09:31:44Z)
• FactGraph: Evaluating Factuality in Summarization with Semantic Graph Representations [114.94628499698096]
  We propose FactGraph, a method that decomposes the document and the summary into structured meaning representations (MRs) MRs describe core semantic concepts and their relations, aggregating the main content in both document and summary in a canonical form, and reducing data sparsity. Experiments on different benchmarks for evaluating factuality show that FactGraph outperforms previous approaches by up to 15%.
  arXiv  Detail & Related papers  (2022-04-13T16:45:33Z)
• Node Feature Extraction by Self-Supervised Multi-scale Neighborhood Prediction [123.20238648121445]
  We propose a new self-supervised learning framework, Graph Information Aided Node feature exTraction (GIANT) GIANT makes use of the eXtreme Multi-label Classification (XMC) formalism, which is crucial for fine-tuning the language model based on graph information. We demonstrate the superior performance of GIANT over the standard GNN pipeline on Open Graph Benchmark datasets.
  arXiv  Detail & Related papers  (2021-10-29T19:55:12Z)
• Are Missing Links Predictable? An Inferential Benchmark for Knowledge Graph Completion [79.07695173192472]
  InferWiki improves upon existing benchmarks in inferential ability, assumptions, and patterns. Each testing sample is predictable with supportive data in the training set. In experiments, we curate two settings of InferWiki varying in sizes and structures, and apply the construction process on CoDEx as comparative datasets.
  arXiv  Detail & Related papers  (2021-08-03T09:51:15Z)
• GraphSVX: Shapley Value Explanations for Graph Neural Networks [81.83769974301995]
  Graph Neural Networks (GNNs) achieve significant performance for various learning tasks on geometric data. In this paper, we propose a unified framework satisfied by most existing GNN explainers. We introduce GraphSVX, a post hoc local model-agnostic explanation method specifically designed for GNNs.
  arXiv  Detail & Related papers  (2021-04-18T10:40:37Z)
• Node Co-occurrence based Graph Neural Networks for Knowledge Graph Link Prediction [13.934907240846197]
  NoKE aims to integrate co-occurrence among entities and relations into graph neural networks to improve knowledge graph completion. NoKE obtains state-of-the-art results on three new, challenging, and difficult benchmark datasets CoDEx for knowledge graph completion.
  arXiv  Detail & Related papers  (2021-04-15T11:51:52Z)
• Message Passing for Hyper-Relational Knowledge Graphs [7.733963597282456]
  We propose a message passing graph encoder - StarE capable of modeling such hyper-relational knowledge graphs. StarE can encode an arbitrary number of additional information (qualifiers) along with the main triple while keeping the semantic roles of qualifiers and triples intact. Our experiments demonstrate that StarE based LP model outperforms existing approaches across multiple benchmarks.
  arXiv  Detail & Related papers  (2020-09-22T22:38:54Z)
• Structure-Augmented Text Representation Learning for Efficient Knowledge Graph Completion [53.31911669146451]
  Human-curated knowledge graphs provide critical supportive information to various natural language processing tasks. These graphs are usually incomplete, urging auto-completion of them. graph embedding approaches, e.g., TransE, learn structured knowledge via representing graph elements into dense embeddings. textual encoding approaches, e.g., KG-BERT, resort to graph triple's text and triple-level contextualized representations.
  arXiv  Detail & Related papers  (2020-04-30T13:50:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.