CANDI: Contextual Alignment for Niche Domains Question Answering

• URL: http://arxiv.org/abs/2607.11891v1
• Date: Wed, 06 May 2026 06:40:13 GMT
• Title: CANDI: Contextual Alignment for Niche Domains Question Answering
• Abstract summary: We introduce CANDI-QA (Contextual Alignment for Niche Domains Question Answering), a novel dataset evaluating large language models (LLMs)<n>We evaluate over ten diverse language models, from compact open-source to state-of-the-art proprietary systems.<n>As a robust baseline, we present MTSS-Net, a lightweight neuro-symbolic framework combining neural retrieval with rule-based reasoning.
• Score: 5.739840108858911
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The deployment of large language models (LLMs) in specialized domains like medical diagnostics and financial advisory necessitates evaluating capabilities beyond general knowledge. Traditional question-answering benchmarks often fail to capture the nuanced contextual grounding, user awareness, and domain understanding these fields require. To address this, we introduce CANDI-QA (Contextual Alignment for Niche Domains Question Answering), a novel dataset evaluating LLMs on delivering accurate, context-sensitive, and user-aligned answers in specialized settings. CANDI-QA features expert-curated question-answer pairs structured into two categories: (1) Information Assistance Questions, which are direct, factual queries requiring precise extraction, and (2) Applied Inference Questions, which are multi-hop reasoning tasks needing situational inference to generate actionable insights. We evaluate over ten diverse language models, from compact open-source to state-of-the-art proprietary systems. As a robust baseline, we present MTSS-Net, a lightweight neuro-symbolic framework combining neural retrieval with rule-based reasoning. Our findings highlight the profound challenges of achieving contextual alignment in niche domains, revealing the limitations of current LLMs without enhanced contextual or symbolic integration. Ultimately, CANDI-QA serves as a critical benchmark for advancing research in context-aware language models, stimulating the development of robust, trustworthy AI for high-stakes domains.

Related papers

• Multimodal QUD: Inquisitive Questions from Scientific Figures [63.41049609329304]
  We generate inquisitive questions that reach the depth of questions humans generate when engaging with scientific papers.<n>We extend the linguistic theory of Questions Under Discussion (QUD) from being text-only to multimodal.<n>We show that fine-tuning a VLM on MQUD shifts the model from generating generic low-level visual questions to content-specific grounding.
  arXiv  Detail & Related papers  (2026-04-26T14:25:23Z)
• Query Expansion in the Age of Pre-trained and Large Language Models: A Comprehensive Survey [21.764997953030857]
  Modern information retrieval must reconcile ambiguous queries with diverse and dynamic corpora.<n>We organize recent work along four complementary dimensions: the point of injection, grounding and interaction, learning and alignment, and knowledge-graph integration.<n>The survey compares traditional and neural QE across seven aspects and maps applications in web search, biomedicine, e-commerce, open-domain question answering/RAG, conversational and code search, and cross-lingual settings.
  arXiv  Detail & Related papers  (2025-09-09T14:31:11Z)
• iQUEST: An Iterative Question-Guided Framework for Knowledge Base Question Answering [9.329800077564673]
  iQUEST is a question-guided KBQA framework that iteratively decomposes complex queries into simpler sub-questions.<n>We integrate a Graph Neural Network (GNN) to look ahead and incorporate 2-hop neighbor information at each reasoning step.
  arXiv  Detail & Related papers  (2025-06-02T15:30:02Z)
• Disambiguation in Conversational Question Answering in the Era of LLMs and Agents: A Survey [54.90240495777929]
  Ambiguity remains a fundamental challenge in Natural Language Processing (NLP)<n>With the advent of Large Language Models (LLMs), addressing ambiguity has become even more critical due to their expanded capabilities and applications.<n>This paper explores the definition, forms, and implications of ambiguity for language driven systems.
  arXiv  Detail & Related papers  (2025-05-18T20:53:41Z)
• Domain-Specific Retrieval-Augmented Generation Using Vector Stores, Knowledge Graphs, and Tensor Factorization [7.522493227357079]
  Large Language Models (LLMs) are pre-trained on large-scale corpora. LLMs suffer from hallucinations, knowledge cut-offs, and lack of knowledge attributions. We introduce SMART-SLIC, a highly domain-specific LLM framework.
  arXiv  Detail & Related papers  (2024-10-03T17:40:55Z)
• Knowledge Tagging with Large Language Model based Multi-Agent System [17.53518487546791]
  This paper investigates the use of a multi-agent system to address the limitations of previous algorithms.<n>We highlight the significant potential of an LLM-based multi-agent system in overcoming the challenges that previous methods have encountered.
  arXiv  Detail & Related papers  (2024-09-12T21:39:01Z)
• Context Matters: Pushing the Boundaries of Open-Ended Answer Generation with Graph-Structured Knowledge Context [4.1229332722825]
  This paper introduces a novel framework that combines graph-driven context retrieval in conjunction to knowledge graphs based enhancement. We conduct experiments on various Large Language Models (LLMs) with different parameter sizes to evaluate their ability to ground knowledge and determine factual accuracy in answers to open-ended questions. Our methodology GraphContextGen consistently outperforms dominant text-based retrieval systems, demonstrating its robustness and adaptability to a larger number of use cases.
  arXiv  Detail & Related papers  (2024-01-23T11:25:34Z)
• DIVKNOWQA: Assessing the Reasoning Ability of LLMs via Open-Domain Question Answering over Knowledge Base and Text [73.68051228972024]
  Large Language Models (LLMs) have exhibited impressive generation capabilities, but they suffer from hallucinations when relying on their internal knowledge. Retrieval-augmented LLMs have emerged as a potential solution to ground LLMs in external knowledge.
  arXiv  Detail & Related papers  (2023-10-31T04:37:57Z)
• Towards Verifiable Generation: A Benchmark for Knowledge-aware Language Model Attribution [48.86322922826514]
  This paper defines a new task of Knowledge-aware Language Model Attribution (KaLMA) First, we extend attribution source from unstructured texts to Knowledge Graph (KG), whose rich structures benefit both the attribution performance and working scenarios. Second, we propose a new Conscious Incompetence" setting considering the incomplete knowledge repository. Third, we propose a comprehensive automatic evaluation metric encompassing text quality, citation quality, and text citation alignment.
  arXiv  Detail & Related papers  (2023-10-09T11:45:59Z)
• Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Survey [100.24095818099522]
  Large language models (LLMs) have significantly advanced the field of natural language processing (NLP) They provide a highly useful, task-agnostic foundation for a wide range of applications. However, directly applying LLMs to solve sophisticated problems in specific domains meets many hurdles.
  arXiv  Detail & Related papers  (2023-05-30T03:00:30Z)
• Question Answering over Knowledge Bases by Leveraging Semantic Parsing and Neuro-Symbolic Reasoning [73.00049753292316]
  We propose a semantic parsing and reasoning-based Neuro-Symbolic Question Answering(NSQA) system. NSQA achieves state-of-the-art performance on QALD-9 and LC-QuAD 1.0.
  arXiv  Detail & Related papers  (2020-12-03T05:17:55Z)
• A Survey on Complex Question Answering over Knowledge Base: Recent Advances and Challenges [71.4531144086568]
  Question Answering (QA) over Knowledge Base (KB) aims to automatically answer natural language questions. Researchers have shifted their attention from simple questions to complex questions, which require more KB triples and constraint inference.
  arXiv  Detail & Related papers  (2020-07-26T07:13:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.