Knowledge Base Completion Meets Transfer Learning

• URL: http://arxiv.org/abs/2108.13073v1
• Date: Mon, 30 Aug 2021 09:13:29 GMT
• Title: Knowledge Base Completion Meets Transfer Learning
• Authors: Vid Kocijan, Thomas Lukasiewicz
• Abstract summary: The aim of knowledge base completion is to predict unseen facts from existing facts in knowledge bases. We introduce the first approach for transfer of knowledge from one collection of facts to another without the need for entity or relation matching.
• Score: 43.89253223499761
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The aim of knowledge base completion is to predict unseen facts from existing facts in knowledge bases. In this work, we introduce the first approach for transfer of knowledge from one collection of facts to another without the need for entity or relation matching. The method works for both canonicalized knowledge bases and uncanonicalized or open knowledge bases, i.e., knowledge bases where more than one copy of a real-world entity or relation may exist. Such knowledge bases are a natural output of automated information extraction tools that extract structured data from unstructured text. Our main contribution is a method that can make use of a large-scale pre-training on facts, collected from unstructured text, to improve predictions on structured data from a specific domain. The introduced method is the most impactful on small datasets such as ReVerb20K, where we obtained 6% absolute increase of mean reciprocal rank and 65% relative decrease of mean rank over the previously best method, despite not relying on large pre-trained models like BERT.

Related papers

• Informed Meta-Learning [55.2480439325792]
  Meta-learning and informed ML stand out as two approaches for incorporating prior knowledge into ML pipelines. We formalise a hybrid paradigm, informed meta-learning, facilitating the incorporation of priors from unstructured knowledge representations. We demonstrate the potential benefits of informed meta-learning in improving data efficiency, robustness to observational noise and task distribution shifts.
  arXiv  Detail & Related papers  (2024-02-25T15:08:37Z)
• Pre-training and Diagnosing Knowledge Base Completion Models [58.07183284468881]
  We introduce and analyze an approach to knowledge transfer from one collection of facts to another without the need for entity or relation matching. The main contribution is a method that can make use of large-scale pre-training on facts, which were collected from unstructured text. To understand the obtained pre-trained models better, we then introduce a novel dataset for the analysis of pre-trained models for Open Knowledge Base Completion.
  arXiv  Detail & Related papers  (2024-01-27T15:20:43Z)
• Utilizing Domain Knowledge: Robust Machine Learning for Building Energy Prediction with Small, Inconsistent Datasets [1.1081836812143175]
  The demand for a huge amount of data for machine learning (ML) applications is currently a bottleneck. We propose a method to combine prior knowledge with data-driven methods to significantly reduce their data dependency. CBML as the knowledge-encoded data-driven method is examined in the context of energy-efficient building engineering.
  arXiv  Detail & Related papers  (2023-01-23T08:56:11Z)
• Schema-aware Reference as Prompt Improves Data-Efficient Knowledge Graph Construction [57.854498238624366]
  We propose a retrieval-augmented approach, which retrieves schema-aware Reference As Prompt (RAP) for data-efficient knowledge graph construction. RAP can dynamically leverage schema and knowledge inherited from human-annotated and weak-supervised data as a prompt for each sample.
  arXiv  Detail & Related papers  (2022-10-19T16:40:28Z)
• Knowledge Enhanced Fine-Tuning for Better Handling Unseen Entities in Dialogue Generation [33.806361531386685]
  We introduce two auxiliary training objectives: 1) Interpret Masked Word, which conjectures the meaning of the masked entity given the context; 2) Hypernym Generation, which predicts the hypernym of the entity based on the context. Experiment results on two dialogue corpus verify the effectiveness of our methods under both knowledge available and unavailable settings.
  arXiv  Detail & Related papers  (2021-09-12T11:13:19Z)
• Information fusion between knowledge and data in Bayesian network structure learning [5.994412766684843]
  This paper describes and evaluates a set of information fusion methods that have been implemented in the open-source Bayesys structure learning system. The results are illustrated both with limited and big data, with application to three BN structure learning algorithms available in Bayesys.
  arXiv  Detail & Related papers  (2021-01-31T15:45:29Z)
• Self-training Improves Pre-training for Natural Language Understanding [63.78927366363178]
  We study self-training as another way to leverage unlabeled data through semi-supervised learning. We introduce SentAugment, a data augmentation method which computes task-specific query embeddings from labeled data. Our approach leads to scalable and effective self-training with improvements of up to 2.6% on standard text classification benchmarks.
  arXiv  Detail & Related papers  (2020-10-05T17:52:25Z)
• Towards a Flexible Embedding Learning Framework [15.604564543883122]
  We propose an embedding learning framework that is flexible in terms of the relationships that can be embedded into the learned representations. A sampling mechanism is carefully designed to establish a direct connection between the input and the information captured by the output embeddings. Our empirical results demonstrate that the proposed framework, in conjunction with a set of relevant entity-relation-matrices, outperforms the existing state-of-the-art approaches in various data mining tasks.
  arXiv  Detail & Related papers  (2020-09-23T08:00:56Z)
• Exploiting Structured Knowledge in Text via Graph-Guided Representation Learning [73.0598186896953]
  We present two self-supervised tasks learning over raw text with the guidance from knowledge graphs. Building upon entity-level masked language models, our first contribution is an entity masking scheme. In contrast to existing paradigms, our approach uses knowledge graphs implicitly, only during pre-training.
  arXiv  Detail & Related papers  (2020-04-29T14:22:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.