HittER: Hierarchical Transformers for Knowledge Graph Embeddings

• URL: http://arxiv.org/abs/2008.12813v2
• Date: Wed, 6 Oct 2021 04:52:07 GMT
• Title: HittER: Hierarchical Transformers for Knowledge Graph Embeddings
• Authors: Sanxing Chen, Xiaodong Liu, Jianfeng Gao, Jian Jiao, Ruofei Zhang and Yangfeng Ji
• Abstract summary: We propose Hitt to learn representations of entities and relations in a complex knowledge graph. Experimental results show that Hitt achieves new state-of-the-art results on multiple link prediction. We additionally propose a simple approach to integrate Hitt into BERT and demonstrate its effectiveness on two Freebase factoid answering datasets.
• Score: 85.93509934018499
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper examines the challenging problem of learning representations of entities and relations in a complex multi-relational knowledge graph. We propose HittER, a Hierarchical Transformer model to jointly learn Entity-relation composition and Relational contextualization based on a source entity's neighborhood. Our proposed model consists of two different Transformer blocks: the bottom block extracts features of each entity-relation pair in the local neighborhood of the source entity and the top block aggregates the relational information from outputs of the bottom block. We further design a masked entity prediction task to balance information from the relational context and the source entity itself. Experimental results show that HittER achieves new state-of-the-art results on multiple link prediction datasets. We additionally propose a simple approach to integrate HittER into BERT and demonstrate its effectiveness on two Freebase factoid question answering datasets.

Related papers

• Joint Multimodal Entity-Relation Extraction Based on Edge-enhanced Graph Alignment Network and Word-pair Relation Tagging [19.872199943795195]
  This paper is the first to propose jointly performing MNER and MRE as a joint multimodal entity-relation extraction task. The proposed method can leverage the edge information to auxiliary alignment between objects and entities.
  arXiv  Detail & Related papers  (2022-11-28T03:23:54Z)
• ReSel: N-ary Relation Extraction from Scientific Text and Tables by Learning to Retrieve and Select [53.071352033539526]
  We study the problem of extracting N-ary relations from scientific articles. Our proposed method ReSel decomposes this task into a two-stage procedure. Our experiments on three scientific information extraction datasets show that ReSel outperforms state-of-the-art baselines significantly.
  arXiv  Detail & Related papers  (2022-10-26T02:28:02Z)
• Informed Multi-context Entity Alignment [27.679124991733907]
  We propose an Informed Multi-context Entity Alignment (IMEA) model to address these issues. In particular, we introduce Transformer to flexibly capture the relation, path, and neighborhood contexts. holistic reasoning is used to estimate alignment probabilities based on both embedding similarity and the relation/entity functionality. Results on several benchmark datasets demonstrate the superiority of our IMEA model compared with existing state-of-the-art entity alignment methods.
  arXiv  Detail & Related papers  (2022-01-02T06:29:30Z)
• EchoEA: Echo Information between Entities and Relations for Entity Alignment [1.1470070927586016]
  We propose a novel framework, Echo Entity Alignment (EchoEA), which leverages self-attention mechanism to spread entity information to relations and echo back to entities. The experimental results on three real-world cross-lingual datasets are stable at around 96% at hits@1 on average.
  arXiv  Detail & Related papers  (2021-07-07T07:34:21Z)
• Enriching Transformers with Structured Tensor-Product Representations for Abstractive Summarization [131.23966358405767]
  We adapt TP-TRANSFORMER with the explicitly compositional Product Representation (TPR) for the task of abstractive summarization. Key feature of our model is a structural bias that we introduce by encoding two separate representations for each token. We show that our TP-TRANSFORMER outperforms the Transformer and the original TP-TRANSFORMER significantly on several abstractive summarization datasets.
  arXiv  Detail & Related papers  (2021-06-02T17:32:33Z)
• RAGA: Relation-aware Graph Attention Networks for Global Entity Alignment [14.287681294725438]
  We propose a novel framework based on Relation-aware Graph Attention Networks to capture the interactions between entities and relations. Our framework adopts the self-attention mechanism to spread entity information to the relations and then aggregate relation information back to entities.
  arXiv  Detail & Related papers  (2021-03-01T06:30:51Z)
• Context-Enhanced Entity and Relation Embedding for Knowledge Graph Completion [2.580765958706854]
  We propose a model named AggrE, which conducts efficient aggregations on entity context and relation context in multi-hops. Experiment results show that AggrE is competitive to existing models.
  arXiv  Detail & Related papers  (2020-12-13T09:20:42Z)
• Pre-training Graph Transformer with Multimodal Side Information for Recommendation [82.4194024706817]
  We propose a pre-training strategy to learn item representations by considering both item side information and their relationships. We develop a novel sampling algorithm named MCNSampling to select contextual neighbors for each item. The proposed Pre-trained Multimodal Graph Transformer (PMGT) learns item representations with two objectives: 1) graph structure reconstruction, and 2) masked node feature reconstruction.
  arXiv  Detail & Related papers  (2020-10-23T10:30:24Z)
• Cross-Supervised Joint-Event-Extraction with Heterogeneous Information Networks [61.950353376870154]
  Joint-event-extraction is a sequence-to-sequence labeling task with a tag set composed of tags of triggers and entities. We propose a Cross-Supervised Mechanism (CSM) to alternately supervise the extraction of triggers or entities. Our approach outperforms the state-of-the-art methods in both entity and trigger extraction.
  arXiv  Detail & Related papers  (2020-10-13T11:51:17Z)
• ENT-DESC: Entity Description Generation by Exploring Knowledge Graph [53.03778194567752]
  In practice, the input knowledge could be more than enough, since the output description may only cover the most significant knowledge. We introduce a large-scale and challenging dataset to facilitate the study of such a practical scenario in KG-to-text. We propose a multi-graph structure that is able to represent the original graph information more comprehensively.
  arXiv  Detail & Related papers  (2020-04-30T14:16:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.