OntologyBench: Can Dense Retrieval Satisfy Structured Biomedical Constraints?

• URL: http://arxiv.org/abs/2609.08174v1
• Date: Tue, 08 Sep 2026 03:04:36 GMT
• Title: OntologyBench: Can Dense Retrieval Satisfy Structured Biomedical Constraints?
• Abstract summary: We introduce OntologyBench, a tiered biomedical retrieval benchmark comprising 471,854 training and 125,744 evaluation query-document relevance pairs.<n>Fine-tuning on ontology-derived supervision improves performance on several relational and compositional tasks.<n>Errors frequently reflect diseases matching only subsets of the phenotype evidence.
• Score: 4.362332162521777
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce OntologyBench, a tiered biomedical retrieval benchmark comprising 471,854 training and 125,744 evaluation query-document relevance pairs across concept grounding, relational retrieval, and compositional phenotype-based retrieval. Although these tasks can be tractable using ontology-aware reference methods, across task tiers, embedding performance is generally lower on relational and compositional tasks than on concept-grounding tasks. Fine-tuning on ontology-derived supervision improves performance on several relational and compositional tasks, whereas the evaluated reranking and LLM-based candidate-scoring methods provide little or no end-to-end improvement. Errors frequently reflect diseases matching only subsets of the phenotype evidence. These findings indicate that the evaluated embedding and reranking configurations do not reliably recover the compatibility encoded by the selected ontology relations and phenotype combinations and motivate retrieval systems that better integrate learned representations with structured biomedical knowledge.

Related papers

• OntologyAligner: Ontology-Aligned Retrieval and Hierarchy-Guided Large Language Model Reranking for Biomedical Ontology Normalization [16.8969067142186]
  We present OntologyAligner, a three-stage framework that combines ontology-aligned retrieval, large language model candidate reranking, and selective hierarchy-guided refinement.<n>OntologyAligner achieved state-of-the-art performance on HPO normalization, with 88.78% Macro Top-1 and 86 Micro Top-1 Accuracy, exceeding the strongest baseline by 4.85 and 5.07 percentage points respectively.
  arXiv  Detail & Related papers  (2026-09-09T11:30:25Z)
• Dementia Etiology Diagnosis via Collaborative Meta Knowledge Enhancement [81.27196033392559]
  We propose a Collaborative Meta Knowledge Enhancement (COME) framework for dementia etiology diagnosis.<n>COME injects multi-center acquisition semantics, source identifiers, and modality indicators as heterogeneity-aware embeddings into a unified Transformer architecture for scale-up training.<n>Our method achieves state-of-the-art in-domain performance with a mean macro-averaged AUC of 85.62% and a 4.29-point gain over the strongest baseline.
  arXiv  Detail & Related papers  (2026-07-24T04:26:47Z)
• Mitigating Shortcut Learning via Feature Disentanglement in Medical Imaging: A Benchmark Study [6.364545942101905]
  Feature disentanglement is a promising approach to mitigate shortcut learning.<n>Shortcut mitigation methods improved classification performance under strong spurious correlations.<n>The best-performing models combine data-centric rebalancing with model-centric disentanglement.
  arXiv  Detail & Related papers  (2026-02-17T10:51:58Z)
• AI Co-Scientist for Knowledge Synthesis in Medical Contexts: A Proof of Concept [0.0]
  We present an AI for scalable and transparent knowledge synthesis based on explicit formalization of Population, Intervention, Comparator, Outcome, and Study design (PICOS)<n>The platform integrates relational storage, vector-based semantic retrieval, and a Neo4j knowledge graph.<n>Results show that PICOS-aware and explainable natural language processing can improve the scalability, transparency, and efficiency of evidence synthesis.
  arXiv  Detail & Related papers  (2026-01-16T23:07:58Z)
• Ontology-Based Concept Distillation for Radiology Report Retrieval and Labeling [10.504309161945065]
  Most existing methods rely on comparing high-dimensional text embeddings from models like CLIP or CXR-BERT.<n>We propose a novel, ontology-driven alternative for comparing radiology report texts based on clinically grounded concepts from the Unified Medical Language System.<n>Our method extracts standardised medical entities from free-text reports using an enhanced pipeline built on RadGraph-XL and SapBERT.
  arXiv  Detail & Related papers  (2025-08-27T14:20:50Z)
• GenOM: Ontology Matching with Description Generation and Large Language Model [19.917106654694894]
  This paper introduces GenOM, a large language model (LLM)-based ontology alignment framework.<n>Experiments conducted on the OAEI Bio-ML track demonstrate that GenOM can often achieve competitive performance.
  arXiv  Detail & Related papers  (2025-08-14T14:48:09Z)
• Pitfalls of topology-aware image segmentation [81.19923502845441]
  We identify critical pitfalls in model evaluation that include inadequate connectivity choices, overlooked topological artifacts, and inappropriate use of evaluation metrics.<n>We propose a set of actionable recommendations to establish fair and robust evaluation standards for topology-aware medical image segmentation methods.
  arXiv  Detail & Related papers  (2024-12-19T08:11:42Z)
• Causal Representation Learning from Multimodal Biomedical Observations [57.00712157758845]
  We develop flexible identification conditions for multimodal data and principled methods to facilitate the understanding of biomedical datasets.<n>Key theoretical contribution is the structural sparsity of causal connections between modalities.<n>Results on a real-world human phenotype dataset are consistent with established biomedical research.
  arXiv  Detail & Related papers  (2024-11-10T16:40:27Z)
• Rethinking Complex Queries on Knowledge Graphs with Neural Link Predictors [58.340159346749964]
  We propose a new neural-symbolic method to support end-to-end learning using complex queries with provable reasoning capability. We develop a new dataset containing ten new types of queries with features that have never been considered. Our method outperforms previous methods significantly in the new dataset and also surpasses previous methods in the existing dataset at the same time.
  arXiv  Detail & Related papers  (2023-04-14T11:35:35Z)
• HiPrompt: Few-Shot Biomedical Knowledge Fusion via Hierarchy-Oriented Prompting [33.1455954220194]
  HiPrompt is a supervision-efficient knowledge fusion framework. It elicits the few-shot reasoning ability of large language models through hierarchy-oriented prompts. Empirical results on the collected KG-Hi-BKF benchmark datasets demonstrate the effectiveness of HiPrompt.
  arXiv  Detail & Related papers  (2023-04-12T16:54:26Z)
• Semantic Search for Large Scale Clinical Ontologies [63.71950996116403]
  We present a deep learning approach to build a search system for large clinical vocabularies. We propose a Triplet-BERT model and a method that generates training data based on semantic training data. The model is evaluated using five real benchmark data sets and the results show that our approach achieves high results on both free text to concept and concept to searching concept vocabularies.
  arXiv  Detail & Related papers  (2022-01-01T05:15:42Z)
• A Diagnostic Study of Explainability Techniques for Text Classification [52.879658637466605]
  We develop a list of diagnostic properties for evaluating existing explainability techniques. We compare the saliency scores assigned by the explainability techniques with human annotations of salient input regions to find relations between a model's performance and the agreement of its rationales with human ones.
  arXiv  Detail & Related papers  (2020-09-25T12:01:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.