Knowledge Graph Refinement based on Triplet BERT-Networks

• URL: http://arxiv.org/abs/2211.10460v1
• Date: Fri, 18 Nov 2022 19:01:21 GMT
• Title: Knowledge Graph Refinement based on Triplet BERT-Networks
• Authors: Armita Khajeh Nassiri (1), Nathalie Pernelle (2), Fatiha Sais (1) and Gianluca Quercini (1) ((1) LISN, CNRS UMR 9015, University of Paris Saclay (2) LIPN, CNRS UMR 7030, University of Sorbonne Paris Nord)
• Abstract summary: This paper adopts a transformer-based triplet network creating an embedding space that clusters the information about an entity or relation in the Knowledge Graph. It creates textual sequences from facts and fine-tunes a triplet network of pre-trained transformer-based language models. We show that GilBERT achieves better or comparable results to the state-of-the-art performance on these two refinement tasks.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Knowledge graph embedding techniques are widely used for knowledge graph refinement tasks such as graph completion and triple classification. These techniques aim at embedding the entities and relations of a Knowledge Graph (KG) in a low dimensional continuous feature space. This paper adopts a transformer-based triplet network creating an embedding space that clusters the information about an entity or relation in the KG. It creates textual sequences from facts and fine-tunes a triplet network of pre-trained transformer-based language models. It adheres to an evaluation paradigm that relies on an efficient spatial semantic search technique. We show that this evaluation protocol is more adapted to a few-shot setting for the relation prediction task. Our proposed GilBERT method is evaluated on triplet classification and relation prediction tasks on multiple well-known benchmark knowledge graphs such as FB13, WN11, and FB15K. We show that GilBERT achieves better or comparable results to the state-of-the-art performance on these two refinement tasks.

Related papers

• Inference over Unseen Entities, Relations and Literals on Knowledge Graphs [1.7474352892977463]
  knowledge graph embedding models have been successfully applied in the transductive setting to tackle various challenging tasks. We propose the attentive byte-pair encoding layer (BytE) to construct a triple embedding from a sequence of byte-pair encoded subword units of entities and relations. BytE leads to massive feature reuse via weight tying, since it forces a knowledge graph embedding model to learn embeddings for subword units instead of entities and relations directly.
  arXiv  Detail & Related papers  (2024-10-09T10:20:54Z)
• A Pure Transformer Pretraining Framework on Text-attributed Graphs [50.833130854272774]
  We introduce a feature-centric pretraining perspective by treating graph structure as a prior. Our framework, Graph Sequence Pretraining with Transformer (GSPT), samples node contexts through random walks. GSPT can be easily adapted to both node classification and link prediction, demonstrating promising empirical success on various datasets.
  arXiv  Detail & Related papers  (2024-06-19T22:30:08Z)
• Few-shot Knowledge Graph Relational Reasoning via Subgraph Adaptation [51.47994645529258]
  Few-shot Knowledge Graph (KG) Reasoning aims to predict unseen triplets (i.e., query triplets) for rare relations in KGs. We propose SAFER (Subgraph Adaptation for Few-shot Reasoning), a novel approach that effectively adapts the information in contextualized graphs to various subgraphs.
  arXiv  Detail & Related papers  (2024-06-19T21:40:35Z)
• Prompt Based Tri-Channel Graph Convolution Neural Network for Aspect Sentiment Triplet Extraction [63.0205418944714]
  Aspect Sentiment Triplet Extraction (ASTE) is an emerging task to extract a given sentence's triplets, which consist of aspects, opinions, and sentiments. Recent studies tend to address this task with a table-filling paradigm, wherein word relations are encoded in a two-dimensional table. We propose a novel model for the ASTE task, called Prompt-based Tri-Channel Graph Convolution Neural Network (PT-GCN), which converts the relation table into a graph to explore more comprehensive relational information.
  arXiv  Detail & Related papers  (2023-12-18T12:46:09Z)
• Exploring Large Language Models for Knowledge Graph Completion [17.139056629060626]
  We consider triples in knowledge graphs as text sequences and introduce an innovative framework called Knowledge Graph LLM. Our technique employs entity and relation descriptions of a triple as prompts and utilizes the response for predictions. Experiments on various benchmark knowledge graphs demonstrate that our method attains state-of-the-art performance in tasks such as triple classification and relation prediction.
  arXiv  Detail & Related papers  (2023-08-26T16:51:17Z)
• ConstGCN: Constrained Transmission-based Graph Convolutional Networks for Document-level Relation Extraction [24.970508961370548]
  Document-level relation extraction with graph neural networks faces a fundamental graph construction gap between training and inference. We propose $textbfConstGCN$, a novel graph convolutional network which performs knowledge-based information propagation between entities. Experimental results show that our method outperforms the previous state-of-the-art (SOTA) approaches on the DocRE dataset.
  arXiv  Detail & Related papers  (2022-10-08T07:36:04Z)
• A Review of Knowledge Graph Completion [0.0]
  Information extraction methods proved to be effective at triple extraction from structured or unstructured data. Most of the current knowledge graphs are incomplete. In order to use KGs in downstream tasks, it is desirable to predict missing links in KGs.
  arXiv  Detail & Related papers  (2022-08-24T16:42:59Z)
• From Discrimination to Generation: Knowledge Graph Completion with Generative Transformer [41.69537736842654]
  We provide an approach GenKGC, which converts knowledge graph completion to sequence-to-sequence generation task with the pre-trained language model. We introduce relation-guided demonstration and entity-aware hierarchical decoding for better representation learning and fast inference. We also release a new large-scale Chinese knowledge graph dataset AliopenKG500 for research purpose.
  arXiv  Detail & Related papers  (2022-02-04T12:52:32Z)
• Structure-Augmented Text Representation Learning for Efficient Knowledge Graph Completion [53.31911669146451]
  Human-curated knowledge graphs provide critical supportive information to various natural language processing tasks. These graphs are usually incomplete, urging auto-completion of them. graph embedding approaches, e.g., TransE, learn structured knowledge via representing graph elements into dense embeddings. textual encoding approaches, e.g., KG-BERT, resort to graph triple's text and triple-level contextualized representations.
  arXiv  Detail & Related papers  (2020-04-30T13:50:34Z)
• Relational Message Passing for Knowledge Graph Completion [78.47976646383222]
  We propose a relational message passing method for knowledge graph completion. It passes relational messages among edges iteratively to aggregate neighborhood information. Results show our method outperforms stateof-the-art knowledge completion methods by a large margin.
  arXiv  Detail & Related papers  (2020-02-17T03:33:41Z)
• Generative Adversarial Zero-Shot Relational Learning for Knowledge Graphs [96.73259297063619]
  We consider a novel formulation, zero-shot learning, to free this cumbersome curation. For newly-added relations, we attempt to learn their semantic features from their text descriptions. We leverage Generative Adrial Networks (GANs) to establish the connection between text and knowledge graph domain.
  arXiv  Detail & Related papers  (2020-01-08T01:19:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.