Contextual Semantic Embeddings for Ontology Subsumption Prediction

• URL: http://arxiv.org/abs/2202.09791v4
• Date: Sat, 18 Mar 2023 18:43:05 GMT
• Title: Contextual Semantic Embeddings for Ontology Subsumption Prediction
• Authors: Jiaoyan Chen and Yuan He and Yuxia Geng and Ernesto Jimenez-Ruiz and Hang Dong and Ian Horrocks
• Abstract summary: We present a new prediction method for contextual embeddings of classes of Web Ontology (OWL) named BERTSubs. It exploits the pre-trained language model BERT to compute embeddings of a class, where customized templates are proposed to incorporate the class context and the logical existential restriction.
• Score: 37.61925808225345
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Automating ontology construction and curation is an important but challenging task in knowledge engineering and artificial intelligence. Prediction by machine learning techniques such as contextual semantic embedding is a promising direction, but the relevant research is still preliminary especially for expressive ontologies in Web Ontology Language (OWL). In this paper, we present a new subsumption prediction method named BERTSubs for classes of OWL ontology. It exploits the pre-trained language model BERT to compute contextual embeddings of a class, where customized templates are proposed to incorporate the class context (e.g., neighbouring classes) and the logical existential restriction. BERTSubs is able to predict multiple kinds of subsumers including named classes from the same ontology or another ontology, and existential restrictions from the same ontology. Extensive evaluation on five real-world ontologies for three different subsumption tasks has shown the effectiveness of the templates and that BERTSubs can dramatically outperform the baselines that use (literal-aware) knowledge graph embeddings, non-contextual word embeddings and the state-of-the-art OWL ontology embeddings.

Related papers

• SciPrompt: Knowledge-augmented Prompting for Fine-grained Categorization of Scientific Topics [2.3742710594744105]
  We introduce SciPrompt, a framework designed to automatically retrieve scientific topic-related terms for low-resource text classification tasks. Our method outperforms state-of-the-art, prompt-based fine-tuning methods on scientific text classification tasks under few and zero-shot settings.
  arXiv  Detail & Related papers  (2024-10-02T18:45:04Z)
• Ontological Relations from Word Embeddings [2.384873896423002]
  It has been reliably shown that the similarity of word embeddings obtained from popular neural models such as BERT approximates effectively a form of semantic similarity of the meaning of those words. We show that a simple feed-forward architecture on top of those embeddings can achieve promising accuracies, with varying generalisation abilities depending on the input data.
  arXiv  Detail & Related papers  (2024-08-01T10:31:32Z)
• Ontology Embedding: A Survey of Methods, Applications and Resources [54.3453925775069]
  Ontologies are widely used for representing domain knowledge and meta data. One straightforward solution is to integrate statistical analysis and machine learning. Numerous papers have been published on embedding, but a lack of systematic reviews hinders researchers from gaining a comprehensive understanding of this field.
  arXiv  Detail & Related papers  (2024-06-16T14:49:19Z)
• Lattice-preserving $\mathcal{ALC}$ ontology embeddings with saturation [50.05281461410368]
  An order-preserving embedding method is proposed to generate embeddings of OWL representations. We show that our method outperforms state-the-art theory-of-the-art embedding methods in several knowledge base completion tasks.
  arXiv  Detail & Related papers  (2023-05-11T22:27:51Z)
• Schema-aware Reference as Prompt Improves Data-Efficient Knowledge Graph Construction [57.854498238624366]
  We propose a retrieval-augmented approach, which retrieves schema-aware Reference As Prompt (RAP) for data-efficient knowledge graph construction. RAP can dynamically leverage schema and knowledge inherited from human-annotated and weak-supervised data as a prompt for each sample.
  arXiv  Detail & Related papers  (2022-10-19T16:40:28Z)
• Knowledge-Aware Bayesian Deep Topic Model [50.58975785318575]
  We propose a Bayesian generative model for incorporating prior domain knowledge into hierarchical topic modeling. Our proposed model efficiently integrates the prior knowledge and improves both hierarchical topic discovery and document representation.
  arXiv  Detail & Related papers  (2022-09-20T09:16:05Z)
• BERTMap: A BERT-based Ontology Alignment System [24.684912604644865]
  BERTMap can support both unsupervised and semi-supervised settings. BERTMap can often perform better than the leading systems LogMap and AML.
  arXiv  Detail & Related papers  (2021-12-05T21:08:44Z)
• OWL2Vec*: Embedding of OWL Ontologies [27.169755467590836]
  We propose a random walk and word embedding based embedding method named OWL2Vec*. OWL2Vec* encodes the semantics of an OWL by taking into account its graph structure, lexical information and logical constructors.
  arXiv  Detail & Related papers  (2020-09-30T13:07:50Z)
• A Diagnostic Study of Explainability Techniques for Text Classification [52.879658637466605]
  We develop a list of diagnostic properties for evaluating existing explainability techniques. We compare the saliency scores assigned by the explainability techniques with human annotations of salient input regions to find relations between a model's performance and the agreement of its rationales with human ones.
  arXiv  Detail & Related papers  (2020-09-25T12:01:53Z)
• Distributional semantic modeling: a revised technique to train term/word vector space models applying the ontology-related approach [36.248702416150124]
  We design a new technique for the distributional semantic modeling with a neural network-based approach to learn distributed term representations (or term embeddings) Vec2graph is a Python library for visualizing word embeddings (term embeddings in our case) as dynamic and interactive graphs.
  arXiv  Detail & Related papers  (2020-03-06T18:27:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.