Representation Learning on Hyper-Relational and Numeric Knowledge Graphs with Transformers

• URL: http://arxiv.org/abs/2305.18256v5
• Date: Sat, 05 Oct 2024 07:26:06 GMT
• Title: Representation Learning on Hyper-Relational and Numeric Knowledge Graphs with Transformers
• Authors: Chanyoung Chung, Jaejun Lee, Joyce Jiyoung Whang,
• Abstract summary: A hyper-relational knowledge graph has been recently studied where a triplet is associated with a set of qualifiers. We propose a unified framework named HyNT that learns representations of a hyper-relational knowledge graph containing numeric literals in either triplets or qualifiers.
• Score: 14.240639250451736
• License:
• Abstract: A hyper-relational knowledge graph has been recently studied where a triplet is associated with a set of qualifiers; a qualifier is composed of a relation and an entity, providing auxiliary information for a triplet. While existing hyper-relational knowledge graph embedding methods assume that the entities are discrete objects, some information should be represented using numeric values, e.g., (J.R.R., was born in, 1892). Also, a triplet (J.R.R., educated at, Oxford Univ.) can be associated with a qualifier such as (start time, 1911). In this paper, we propose a unified framework named HyNT that learns representations of a hyper-relational knowledge graph containing numeric literals in either triplets or qualifiers. We define a context transformer and a prediction transformer to learn the representations based not only on the correlations between a triplet and its qualifiers but also on the numeric information. By learning compact representations of triplets and qualifiers and feeding them into the transformers, we reduce the computation cost of using transformers. Using HyNT, we can predict missing numeric values in addition to missing entities or relations in a hyper-relational knowledge graph. Experimental results show that HyNT significantly outperforms state-of-the-art methods on real-world datasets.

Related papers

• Inference over Unseen Entities, Relations and Literals on Knowledge Graphs [1.7474352892977463]
  knowledge graph embedding models have been successfully applied in the transductive setting to tackle various challenging tasks. We propose the attentive byte-pair encoding layer (BytE) to construct a triple embedding from a sequence of byte-pair encoded subword units of entities and relations. BytE leads to massive feature reuse via weight tying, since it forces a knowledge graph embedding model to learn embeddings for subword units instead of entities and relations directly.
  arXiv  Detail & Related papers  (2024-10-09T10:20:54Z)
• Enhanced Data Transfer Cooperating with Artificial Triplets for Scene Graph Generation [15.109087477826106]
  This work focuses on training dataset enhancement of informative relational triplets for Scene Graph Generation (SGG) We propose two novel training dataset enhancement modules: Feature Space Triplet Augmentation (FSTA) and Soft Transfer. Experimental results show that integrating FSTA and Soft Transfer achieve high levels of both Recall and mean Recall in Visual Genome dataset.
  arXiv  Detail & Related papers  (2024-06-27T16:52:01Z)
• Graph Relation Distillation for Efficient Biomedical Instance Segmentation [80.51124447333493]
  We propose a graph relation distillation approach for efficient biomedical instance segmentation. We introduce two graph distillation schemes deployed at both the intra-image level and the inter-image level. Experimental results on a number of biomedical datasets validate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2024-01-12T04:41:23Z)
• A Dataset for Hyper-Relational Extraction and a Cube-Filling Approach [59.89749342550104]
  We propose the task of hyper-relational extraction to extract more specific and complete facts from text. Existing models cannot perform hyper-relational extraction as it requires a model to consider the interaction between three entities. We propose CubeRE, a cube-filling model inspired by table-filling approaches and explicitly considers the interaction between relation triplets and qualifiers.
  arXiv  Detail & Related papers  (2022-11-18T03:51:28Z)
• Learning Representations for Hyper-Relational Knowledge Graphs [35.380689788802776]
  We design a framework to learn representations for hyper-relational facts using multiple aggregators. Experiments demonstrate the effectiveness of our framework across multiple datasets. We conduct an ablation study that validates the importance of the various components in our framework.
  arXiv  Detail & Related papers  (2022-08-30T15:02:14Z)
• Repurposing Knowledge Graph Embeddings for Triple Representation via Weak Supervision [77.34726150561087]
  Current methods learn triple embeddings from scratch without utilizing entity and predicate embeddings from pre-trained models. We develop a method for automatically sampling triples from a knowledge graph and estimating their pairwise similarities from pre-trained embedding models. These pairwise similarity scores are then fed to a Siamese-like neural architecture to fine-tune triple representations.
  arXiv  Detail & Related papers  (2022-08-22T14:07:08Z)
• Relphormer: Relational Graph Transformer for Knowledge Graph Representations [25.40961076988176]
  We propose a new variant of Transformer for knowledge graph representations dubbed Relphormer. We propose a novel structure-enhanced self-attention mechanism to encode the relational information and keep the semantic information within entities and relations. Experimental results on six datasets show that Relphormer can obtain better performance compared with baselines.
  arXiv  Detail & Related papers  (2022-05-22T15:30:18Z)
• An Adversarial Transfer Network for Knowledge Representation Learning [11.013390624382257]
  We propose an adversarial embedding transfer network ATransN, which transfers knowledge from one or more teacher knowledge graphs to a target one. Specifically, we add soft constraints on aligned entity pairs and neighbours to the existing knowledge representation learning methods.
  arXiv  Detail & Related papers  (2021-04-30T05:07:25Z)
• Link Prediction on N-ary Relational Data Based on Relatedness Evaluation [61.61555159755858]
  We propose a method called NaLP to conduct link prediction on n-ary relational data. We represent each n-ary relational fact as a set of its role and role-value pairs. Experimental results validate the effectiveness and merits of the proposed methods.
  arXiv  Detail & Related papers  (2021-04-21T09:06:54Z)
• HittER: Hierarchical Transformers for Knowledge Graph Embeddings [85.93509934018499]
  We propose Hitt to learn representations of entities and relations in a complex knowledge graph. Experimental results show that Hitt achieves new state-of-the-art results on multiple link prediction. We additionally propose a simple approach to integrate Hitt into BERT and demonstrate its effectiveness on two Freebase factoid answering datasets.
  arXiv  Detail & Related papers  (2020-08-28T18:58:15Z)
• Structure-Augmented Text Representation Learning for Efficient Knowledge Graph Completion [53.31911669146451]
  Human-curated knowledge graphs provide critical supportive information to various natural language processing tasks. These graphs are usually incomplete, urging auto-completion of them. graph embedding approaches, e.g., TransE, learn structured knowledge via representing graph elements into dense embeddings. textual encoding approaches, e.g., KG-BERT, resort to graph triple's text and triple-level contextualized representations.
  arXiv  Detail & Related papers  (2020-04-30T13:50:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.