Do General NLP Embeddings Capture Ontological Reasoning?

• URL: http://arxiv.org/abs/2609.00177v1
• Date: Mon, 31 Aug 2026 18:03:23 GMT
• Title: Do General NLP Embeddings Capture Ontological Reasoning?
• Abstract summary: General-purpose NLP embedding models perform well on linguistic tasks, but their ability to capture symbolic ontological inversions remains unclear.<n>We evaluate more than 25 state-of-the-art embedding models and find substantial limitations.<n>Fine-tuning improves discrimination by a large margin but transfers poorly to downstream Semantic Web tasks.
• Score: 5.477826608330914
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: General-purpose NLP embedding models perform well on linguistic tasks, but their ability to capture symbolic ontological structure remains unclear. We introduce AVA, a systematic framework for evaluating whether embeddings distinguish logic-sensitive relational semantics in ontologies and knowledge graphs. AVA comprises 171,007 contrastive triplets derived from 163 heterogeneous ontologies using hierarchy inversion, relation substitution, and disjointness injection. Each triplet contains an ontology statement, a semantically equivalent paraphrase, and a logic-sensitive hard negative with contradictory relational meaning. We evaluate more than 25 state-of-the-art embedding models and find substantial limitations: the best model achieves only 0.739 triplet accuracy, while hard negative accuracy falls to 0.135. Fine-tuning improves discrimination by a large margin but transfers poorly to downstream Semantic Web tasks, including taxonomy discovery and ontology alignment. Further analysis suggests that improvements stem partly from perturbation-specific pattern recognition rather than robust ontological understanding. These findings reveal a persistent gap between linguistic representation learning and ontology-level discrimination, challenging the assumption that strong NLP benchmark performance translates to Semantic Web competence.

Related papers

• Structured-Condensed Prompt Tuning in Vision-Language Models for Fine-grained Image Recognition [38.294322671641545]
  Fine-grained image recognition is a significant challenge due to the substantial expertise and effort required for manual annotation.<n>Current methods treat class labels as isolated, discrete entities, overlooking the rich semantic relationships between them.<n>We propose Structured-Condensed Prompt Tuning ( SCPT), which enhances semantic structure modeling in prompt learning.
  arXiv  Detail & Related papers  (2026-07-07T12:09:49Z)
• Mastering Negation: Boosting Grounding Models via Grouped Opposition-Based Learning [87.15765427638195]
  We introduce D-Negation, a new dataset that provides objects annotated with both positive and negative semantic descriptions.<n>We propose a grouped opposition-based learning framework that learns negation-aware representations from limited samples.<n>By fine-tuning fewer than 10 percent of the model parameters, our approach achieves improvements of up to 4.4 mAP and 5.7 mAP on positive and negative semantic evaluations.
  arXiv  Detail & Related papers  (2026-03-13T03:21:48Z)
• Generalization Gaps in Political Fake News Detection: An Empirical Study on the LIAR Dataset [0.764671395172401]
  We present a diagnostic evaluation of nine machine learning algorithms on the LIAR benchmark.<n>We uncover a hard "Performance Ceiling", with fine-grained classification not exceeding a weighted F1-score of 0.32 across models.<n>A massive "Generalization Gap" in tree-based ensembles, which achieve more than 99% training accuracy but collapse to approximately 25% on test data.
  arXiv  Detail & Related papers  (2025-12-20T23:08:18Z)
• Sensitivity of Small Language Models to Fine-tuning Data Contamination [0.0]
  Small Language Models (SLMs) are increasingly being deployed in resource-constrained environments.<n>We measure susceptibility to syntactic and semantic transformation types during instruction tuning.<n>Character reversal produces near-complete failure across all models regardless of size or family.<n>Semantic transformations demonstrate distinct threshold behaviors and greater resilience in core linguistic capabilities.
  arXiv  Detail & Related papers  (2025-11-10T06:44:29Z)
• TopInG: Topologically Interpretable Graph Learning via Persistent Rationale Filtration [10.830399323047265]
  We propose TopInG: Topologically Interpretable Graph Learning, a novel framework to identify persistent rationale subgraphs.<n>TopInG employs a rationale filtration learning approach to model an autoregressive generation process of rationale subgraphs.<n>Our approach improves upon state-of-the-art methods on both predictive accuracy and interpretation quality.
  arXiv  Detail & Related papers  (2025-10-06T17:59:44Z)
• RealUnify: Do Unified Models Truly Benefit from Unification? A Comprehensive Benchmark [71.3555284685426]
  We introduce RealUnify, a benchmark designed to evaluate bidirectional capability synergy.<n>RealUnify comprises 1,000 meticulously human-annotated instances spanning 10 categories and 32 subtasks.<n>We find that current unified models still struggle to achieve effective synergy, indicating that architectural unification alone is insufficient.
  arXiv  Detail & Related papers  (2025-09-29T15:07:28Z)
• Hierarchical Invariance for Robust and Interpretable Vision Tasks at Larger Scales [54.78115855552886]
  We show how to construct over-complete invariants with a Convolutional Neural Networks (CNN)-like hierarchical architecture. With the over-completeness, discriminative features w.r.t. the task can be adaptively formed in a Neural Architecture Search (NAS)-like manner. For robust and interpretable vision tasks at larger scales, hierarchical invariant representation can be considered as an effective alternative to traditional CNN and invariants.
  arXiv  Detail & Related papers  (2024-02-23T16:50:07Z)
• Semantic Sensitivities and Inconsistent Predictions: Measuring the Fragility of NLI Models [44.56781176879151]
  State-of-the-art Natural Language Inference (NLI) models are sensitive towards minor semantics preserving surface-form variations. We show that semantic sensitivity causes performance degradations of $12.92%$ and $23.71%$ average over $textbfin-$ and $textbfout-of-$ domain settings.
  arXiv  Detail & Related papers  (2024-01-25T14:47:05Z)
• AS-XAI: Self-supervised Automatic Semantic Interpretation for CNN [5.42467030980398]
  We propose a self-supervised automatic semantic interpretable artificial intelligence (AS-XAI) framework. It utilizes transparent embedding semantic extraction spaces and row-centered principal component analysis (PCA) for global semantic interpretation of model decisions. The proposed approach offers broad fine-grained practical applications, including shared semantic interpretation under out-of-distribution categories.
  arXiv  Detail & Related papers  (2023-12-02T10:06:54Z)
• Logic-induced Diagnostic Reasoning for Semi-supervised Semantic Segmentation [85.12429517510311]
  LogicDiag is a neural-logic semi-supervised learning framework for semantic segmentation. Our key insight is that conflicts within pseudo labels, identified through symbolic knowledge, can serve as strong yet commonly ignored learning signals. We showcase the practical application of LogicDiag in the data-hungry segmentation scenario, where we formalize the structured abstraction of semantic concepts as a set of logic rules.
  arXiv  Detail & Related papers  (2023-08-24T06:50:07Z)
• STAR Loss: Reducing Semantic Ambiguity in Facial Landmark Detection [80.04000067312428]
  We propose a Self-adapTive Ambiguity Reduction (STAR) loss by exploiting the properties of semantic ambiguity. We find that semantic ambiguity results in the anisotropic predicted distribution, which inspires us to use predicted distribution to represent semantic ambiguity. We also propose two kinds of eigenvalue restriction methods that could avoid both distribution's abnormal change and the model's premature convergence.
  arXiv  Detail & Related papers  (2023-06-05T10:33:25Z)
• Leveraging Unlabeled Data for Entity-Relation Extraction through Probabilistic Constraint Satisfaction [54.06292969184476]
  We study the problem of entity-relation extraction in the presence of symbolic domain knowledge. Our approach employs semantic loss which captures the precise meaning of a logical sentence. With a focus on low-data regimes, we show that semantic loss outperforms the baselines by a wide margin.
  arXiv  Detail & Related papers  (2021-03-20T00:16:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.