Language Models as Ontology Encoders

• URL: http://arxiv.org/abs/2507.14334v1
• Date: Fri, 18 Jul 2025 19:26:16 GMT
• Title: Language Models as Ontology Encoders
• Authors: Hui Yang, Jiaoyan Chen, Yuan He, Yongsheng Gao, Ian Horrocks,
• Abstract summary: Ontology embeddings can infer plausible new knowledge and approximate complex reasoning.<n>OnT tunes a Pretrained Model Language (PLM) via incorporating hyperbolic modeling in a geometric space.<n>OnT consistently outperforms the baselines in both tasks of prediction and inference.
• Score: 32.148744398729896
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: OWL (Web Ontology Language) ontologies which are able to formally represent complex knowledge and support semantic reasoning have been widely adopted across various domains such as healthcare and bioinformatics. Recently, ontology embeddings have gained wide attention due to its potential to infer plausible new knowledge and approximate complex reasoning. However, existing methods face notable limitations: geometric model-based embeddings typically overlook valuable textual information, resulting in suboptimal performance, while the approaches that incorporate text, which are often based on language models, fail to preserve the logical structure. In this work, we propose a new ontology embedding method OnT, which tunes a Pretrained Language Model (PLM) via geometric modeling in a hyperbolic space for effectively incorporating textual labels and simultaneously preserving class hierarchies and other logical relationships of Description Logic EL. Extensive experiments on four real-world ontologies show that OnT consistently outperforms the baselines including the state-of-the-art across both tasks of prediction and inference of axioms. OnT also demonstrates strong potential in real-world applications, indicated by its robust transfer learning abilities and effectiveness in real cases of constructing a new ontology from SNOMED CT. Data and code are available at https://github.com/HuiYang1997/OnT.

Related papers

• TransBox: EL++-closed Ontology Embedding [14.850996103983187]
  We develop an effective EL++-closed embedding method that can handle many-to-one, one-to-many and many-to-many relations.<n>Our experiments demonstrate that TransBox achieves state-of-the-art performance across various real-world datasets for predicting complex axioms.
  arXiv  Detail & Related papers  (2024-10-18T16:17:10Z)
• Ontological Relations from Word Embeddings [2.384873896423002]
  It has been reliably shown that the similarity of word embeddings obtained from popular neural models such as BERT approximates effectively a form of semantic similarity of the meaning of those words. We show that a simple feed-forward architecture on top of those embeddings can achieve promising accuracies, with varying generalisation abilities depending on the input data.
  arXiv  Detail & Related papers  (2024-08-01T10:31:32Z)
• Ontology Embedding: A Survey of Methods, Applications and Resources [54.3453925775069]
  Onologies are widely used for representing domain knowledge and meta data.<n> logical reasoning that can directly support are quite limited in learning, approximation and prediction.<n>One straightforward solution is to integrate statistical analysis and machine learning.
  arXiv  Detail & Related papers  (2024-06-16T14:49:19Z)
• Position: Topological Deep Learning is the New Frontier for Relational Learning [51.05869778335334]
  Topological deep learning (TDL) is a rapidly evolving field that uses topological features to understand and design deep learning models. This paper posits that TDL is the new frontier for relational learning.
  arXiv  Detail & Related papers  (2024-02-14T00:35:10Z)
• How Well Do Text Embedding Models Understand Syntax? [50.440590035493074]
  The ability of text embedding models to generalize across a wide range of syntactic contexts remains under-explored. Our findings reveal that existing text embedding models have not sufficiently addressed these syntactic understanding challenges. We propose strategies to augment the generalization ability of text embedding models in diverse syntactic scenarios.
  arXiv  Detail & Related papers  (2023-11-14T08:51:00Z)
• LOGICSEG: Parsing Visual Semantics with Neural Logic Learning and Reasoning [73.98142349171552]
  LOGICSEG is a holistic visual semantic that integrates neural inductive learning and logic reasoning with both rich data and symbolic knowledge. During fuzzy logic-based continuous relaxation, logical formulae are grounded onto data and neural computational graphs, hence enabling logic-induced network training. These designs together make LOGICSEG a general and compact neural-logic machine that is readily integrated into existing segmentation models.
  arXiv  Detail & Related papers  (2023-09-24T05:43:19Z)
• Constructing Word-Context-Coupled Space Aligned with Associative Knowledge Relations for Interpretable Language Modeling [0.0]
  The black-box structure of the deep neural network in pre-trained language models seriously limits the interpretability of the language modeling process. A Word-Context-Coupled Space (W2CSpace) is proposed by introducing the alignment processing between uninterpretable neural representation and interpretable statistical logic. Our language model can achieve better performance and highly credible interpretable ability compared to related state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-05-19T09:26:02Z)
• Dual Box Embeddings for the Description Logic EL++ [16.70961576041243]
  Similar to Knowledge Graphs (KGs), Knowledge Graphs are often incomplete, and maintaining and constructing them has proved challenging. Similar to KGs, a promising approach is to learn embeddings in a latent vector space, while additionally ensuring they adhere to the semantics of the underlying DL. We propose a novel ontology embedding method named Box$2$EL for the DL EL++, which represents both concepts and roles as boxes.
  arXiv  Detail & Related papers  (2023-01-26T14:13:37Z)
• Contextual Semantic Embeddings for Ontology Subsumption Prediction [37.61925808225345]
  We present a new prediction method for contextual embeddings of classes of Web Ontology (OWL) named BERTSubs. It exploits the pre-trained language model BERT to compute embeddings of a class, where customized templates are proposed to incorporate the class context and the logical existential restriction.
  arXiv  Detail & Related papers  (2022-02-20T11:14:04Z)
• Logical Natural Language Generation from Open-Domain Tables [107.04385677577862]
  We propose a new task where a model is tasked with generating natural language statements that can be emphlogically entailed by the facts. To facilitate the study of the proposed logical NLG problem, we use the existing TabFact dataset citechen 2019tabfact featured with a wide range of logical/symbolic inferences. The new task poses challenges to the existing monotonic generation frameworks due to the mismatch between sequence order and logical order.
  arXiv  Detail & Related papers  (2020-04-22T06:03:10Z)
• How Far are We from Effective Context Modeling? An Exploratory Study on Semantic Parsing in Context [59.13515950353125]
  We present a grammar-based decoding semantic parsing and adapt typical context modeling methods on top of it. We evaluate 13 context modeling methods on two large cross-domain datasets, and our best model achieves state-of-the-art performances.
  arXiv  Detail & Related papers  (2020-02-03T11:28:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.