When CQs Go Wrong: Challenges in CQ Verification with OE-Assist

• URL: http://arxiv.org/abs/2606.24619v1
• Date: Tue, 23 Jun 2026 14:17:14 GMT
• Title: When CQs Go Wrong: Challenges in CQ Verification with OE-Assist
• Abstract summary: This paper investigates what makes a Competency Questions (CQs) challenging and possible solutions to enhance the users' performance in the CQ-verification process.<n>The results show the necessity of a tool to refine CQs before publishing them to avoid ambiguity or excessive complexity.
• Score: 1.451926072497166
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Competency Questions (CQs) are the central component of CQ-verification, an established process in which an ontology is evaluated against a set of natural language questions to determine whether the intended purpose of the ontology has been properly modelled. However, CQ-verification is often time-consuming and error-prone, as it requires careful interpretation of linguistic nuances and precise alignment with formal ontology constructs. Ambiguities and complexity in CQs can further complicate this process, leading to inconsistent modelling decisions and verification outcomes. In this paper, we investigate what makes a CQ challenging and possible solutions to enhance the users' performance in the CQ-verification process. We experimented with the data of 19 participants who performed CQ-verification on 20 tasks using an LLM assistant to support ontology evaluation. The results show the necessity of a tool to refine CQs before publishing them to avoid ambiguity or excessive complexity in later phases of the ontology engineering process.

Related papers

• Characterising LLM-Generated Competency Questions: a Cross-Domain Empirical Study using Open and Closed Models [0.11363776709575053]
  This paper introduces measures for the systematic comparison of Competency Questions (CQs) across multiple dimensions.<n>We identify their salient properties, including readability, relevance with respect to the input text and structural complexity of the generated questions.
  arXiv  Detail & Related papers  (2026-04-17T17:16:32Z)
• AgenticIQA: An Agentic Framework for Adaptive and Interpretable Image Quality Assessment [69.06977852423564]
  Image quality assessment (IQA) reflects both the quantification and interpretation of perceptual quality rooted in the human visual system.<n>AgenticIQA decomposes IQA into four subtasks -- distortion detection, distortion analysis, tool selection, and tool execution.<n>To support training and evaluation, we introduce AgenticIQA-200K, a large-scale instruction dataset tailored for IQA agents, and AgenticIQA-Eval, the first benchmark for assessing the planning, execution, and summarization capabilities of VLM-based IQA agents.
  arXiv  Detail & Related papers  (2025-09-30T09:37:01Z)
• A Comparative Study of Competency Question Elicitation Methods from Ontology Requirements [0.22499166814992444]
  This paper presents an empirical evaluation of three distinct CQ formulation approaches.<n>We generate CQs using each approach from a set of requirements for cultural heritage.<n>We assess them across different dimensions: degree of acceptability, ambiguity, relevance, complexity and readability.
  arXiv  Detail & Related papers  (2025-07-01T14:49:30Z)
• Discerning and Characterising Types of Competency Questions for Ontologies [0.4757470449749875]
  Competency Questions (CQs) are widely used in ontology development by guiding, among others, the scoping and validation stages.<n>Very limited guidance exists for formulating CQs and assessing whether they are good CQs, leading to issues such as ambiguity and unusable formulations.<n>This paper contributes to such theoretical foundations by analysing questions, their uses, and the myriad of development tasks.
  arXiv  Detail & Related papers  (2024-12-18T10:26:29Z)
• Few-Shot Image Quality Assessment via Adaptation of Vision-Language Models [93.91086467402323]
  Gradient-Regulated Meta-Prompt IQA Framework (GRMP-IQA) designed to efficiently adapt the visual-language pre-trained model, CLIP, to IQA tasks.<n> GRMP-IQA consists of two core modules: (i) Meta-Prompt Pre-training Module and (ii) Quality-Aware Gradient Regularization.
  arXiv  Detail & Related papers  (2024-09-09T07:26:21Z)
• UniOQA: A Unified Framework for Knowledge Graph Question Answering with Large Language Models [4.627548680442906]
  OwnThink is the most extensive Chinese open-domain knowledge graph introduced in recent times. We introduce UniOQA, a unified framework that integrates two parallel approaches to question answering. UniOQA notably advances SpCQL Logical Accuracy to 21.2% and Execution Accuracy to 54.9%, achieving the new state-of-the-art results on this benchmark.
  arXiv  Detail & Related papers  (2024-06-04T08:36:39Z)
• An Experiment in Retrofitting Competency Questions for Existing Ontologies [0.0]
  Inspecting CQs together with the axioms provides critical insights into the scope and applicability of the CQs. CQs are integral to the majority of engineering methodologies, but the practice of publishing CQs alongside the on artefacts is not widely observed.
  arXiv  Detail & Related papers  (2023-11-09T08:57:39Z)
• QADYNAMICS: Training Dynamics-Driven Synthetic QA Diagnostic for Zero-Shot Commonsense Question Answering [48.25449258017601]
  State-of-the-art approaches fine-tune language models on QA pairs constructed from CommonSense Knowledge Bases. We propose QADYNAMICS, a training dynamics-driven framework for QA diagnostics and refinement.
  arXiv  Detail & Related papers  (2023-10-17T14:27:34Z)
• Matching Game for Optimized Association in Quantum Communication Networks [65.16483325184237]
  This paper proposes a swap-stable request-QS association algorithm for quantum switches. It achieves a near-optimal (within 5%) performance in terms of the percentage of served requests. It is shown to be scalable and maintain its near-optimal performance even when the size of the QCN increases.
  arXiv  Detail & Related papers  (2023-05-22T03:39:18Z)
• ProQA: Structural Prompt-based Pre-training for Unified Question Answering [84.59636806421204]
  ProQA is a unified QA paradigm that solves various tasks through a single model. It concurrently models the knowledge generalization for all QA tasks while keeping the knowledge customization for every specific QA task. ProQA consistently boosts performance on both full data fine-tuning, few-shot learning, and zero-shot testing scenarios.
  arXiv  Detail & Related papers  (2022-05-09T04:59:26Z)
• Conditional Self-Attention for Query-based Summarization [49.616774159367516]
  We propose textitconditional self-attention (CSA), a neural network module designed for conditional dependency modeling. Experiments on Debatepedia and HotpotQA benchmark datasets show CSA consistently outperforms vanilla Transformer.
  arXiv  Detail & Related papers  (2020-02-18T02:22:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.