KERMIT - A Transformer-Based Approach for Knowledge Graph Matching

• URL: http://arxiv.org/abs/2204.13931v1
• Date: Fri, 29 Apr 2022 08:07:17 GMT
• Title: KERMIT - A Transformer-Based Approach for Knowledge Graph Matching
• Authors: Sven Hertling, Jan Portisch, Heiko Paulheim
• Abstract summary: One of the strongest signals for automated matching of knowledge graphs and textual concept descriptions are concept descriptions. We show that performing pairwise comparisons of all textual descriptions of concepts in two knowledge graphs is expensive and scales quadratically. We first generate matching candidates using a pre-trained sentence transformer. In a second step, we use fine-tuned transformer cross-encoders to generate the best candidates.
• Score: 1.9981375888949477
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: One of the strongest signals for automated matching of knowledge graphs and ontologies are textual concept descriptions. With the rise of transformer-based language models, text comparison based on meaning (rather than lexical features) is available to researchers. However, performing pairwise comparisons of all textual descriptions of concepts in two knowledge graphs is expensive and scales quadratically (or even worse if concepts have more than one description). To overcome this problem, we follow a two-step approach: we first generate matching candidates using a pre-trained sentence transformer (so called bi-encoder). In a second step, we use fine-tuned transformer cross-encoders to generate the best candidates. We evaluate our approach on multiple datasets and show that it is feasible and produces competitive results.

Related papers

• Evaluating Semantic Variation in Text-to-Image Synthesis: A Causal Perspective [50.261681681643076]
  We propose a novel metric called SemVarEffect and a benchmark named SemVarBench to evaluate the causality between semantic variations in inputs and outputs in text-to-image synthesis. Our work establishes an effective evaluation framework that advances the T2I synthesis community's exploration of human instruction understanding.
  arXiv  Detail & Related papers  (2024-10-14T08:45:35Z)
• Sentiment analysis in Tourism: Fine-tuning BERT or sentence embeddings concatenation? [0.0]
  We conduct a comparative study between Fine-Tuning the Bidirectional Representations from Transformers and a method of concatenating two embeddings to boost the performance of a stacked Bidirectional Long Short-Term Memory-Bidirectional Gated Recurrent Units model. A search for the best learning rate was made at the level of the two approaches, and a comparison of the best embeddings was made for each sentence embedding combination.
  arXiv  Detail & Related papers  (2023-12-12T23:23:23Z)
• Pure Transformers are Powerful Graph Learners [51.36884247453605]
  We show that standard Transformers without graph-specific modifications can lead to promising results in graph learning both in theory and practice. We prove that this approach is theoretically at least as expressive as an invariant graph network (2-IGN) composed of equivariant linear layers. Our method coined Tokenized Graph Transformer (TokenGT) achieves significantly better results compared to GNN baselines and competitive results.
  arXiv  Detail & Related papers  (2022-07-06T08:13:06Z)
• Relphormer: Relational Graph Transformer for Knowledge Graph Representations [25.40961076988176]
  We propose a new variant of Transformer for knowledge graph representations dubbed Relphormer. We propose a novel structure-enhanced self-attention mechanism to encode the relational information and keep the semantic information within entities and relations. Experimental results on six datasets show that Relphormer can obtain better performance compared with baselines.
  arXiv  Detail & Related papers  (2022-05-22T15:30:18Z)
• SepTr: Separable Transformer for Audio Spectrogram Processing [74.41172054754928]
  We propose a new vision transformer architecture called Separable Transformer (SepTr) SepTr employs two transformer blocks in a sequential manner, the first attending to tokens within the same frequency bin, and the second attending to tokens within the same time interval. We conduct experiments on three benchmark data sets, showing that our architecture outperforms conventional vision transformers and other state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-03-17T19:48:43Z)
• Hierarchical Sketch Induction for Paraphrase Generation [79.87892048285819]
  We introduce Hierarchical Refinement Quantized Variational Autoencoders (HRQ-VAE), a method for learning decompositions of dense encodings. We use HRQ-VAE to encode the syntactic form of an input sentence as a path through the hierarchy, allowing us to more easily predict syntactic sketches at test time.
  arXiv  Detail & Related papers  (2022-03-07T15:28:36Z)
• Unleashing the Power of Transformer for Graphs [28.750700720796836]
  Transformer suffers from the scalability problem when dealing with graphs. We propose a new Transformer architecture, named dual-encoding Transformer (DET) DET has a structural encoder to aggregate information from connected neighbors and a semantic encoder to focus on semantically useful distant nodes.
  arXiv  Detail & Related papers  (2022-02-18T06:40:51Z)
• Matching with Transformers in MELT [1.2891210250935146]
  We provide an easy to use implementation in the MELT framework which is suited for ontology and knowledge graph matching. We show that a transformer-based filter helps to choose the correct correspondences given a high-recall alignment.
  arXiv  Detail & Related papers  (2021-09-15T16:07:43Z)
• Transformer Based Language Models for Similar Text Retrieval and Ranking [0.0]
  We introduce novel approaches for effectively applying neural transformer models to similar text retrieval and ranking. By eliminating the bag-of-words-based step, our approach is able to accurately retrieve and rank results even when they have no non-stopwords in common with the query.
  arXiv  Detail & Related papers  (2020-05-10T06:12:53Z)
• Structure-Augmented Text Representation Learning for Efficient Knowledge Graph Completion [53.31911669146451]
  Human-curated knowledge graphs provide critical supportive information to various natural language processing tasks. These graphs are usually incomplete, urging auto-completion of them. graph embedding approaches, e.g., TransE, learn structured knowledge via representing graph elements into dense embeddings. textual encoding approaches, e.g., KG-BERT, resort to graph triple's text and triple-level contextualized representations.
  arXiv  Detail & Related papers  (2020-04-30T13:50:34Z)
• Auto-Encoding Twin-Bottleneck Hashing [141.5378966676885]
  This paper proposes an efficient and adaptive code-driven graph. It is updated by decoding in the context of an auto-encoder. Experiments on benchmarked datasets clearly show the superiority of our framework over the state-of-the-art hashing methods.
  arXiv  Detail & Related papers  (2020-02-27T05:58:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.