An Evidence-First Multi-LLM Framework for Auditable Critical-Infrastructure Dependency Modeling

• URL: http://arxiv.org/abs/2609.12360v1
• Date: Fri, 11 Sep 2026 02:33:13 GMT
• Title: An Evidence-First Multi-LLM Framework for Auditable Critical-Infrastructure Dependency Modeling
• Abstract summary: Critical-infrastructure knowledge is distributed across heterogeneous, incomplete, and weakly structured evidence.<n>We present an evidence-first multi-LLM framework for constructing Infrastructure Knowledge Bases (IKBs) and Infrastructure Dependency Graphs (IDGs) from heterogeneous infrastructure documentation.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Critical-infrastructure knowledge is distributed across heterogeneous, incomplete, and weakly structured evidence, making dependency models difficult to construct automatically and difficult to trust. Large language models (LLMs) can extract structured knowledge from such evidence, but direct LLM-to-graph generation risks unsupported relationships, inconsistent terminology, incorrect entity identities, and erroneous dependency endpoints. We present an evidence-first multi-LLM framework for constructing Infrastructure Knowledge Bases (IKBs) and Infrastructure Dependency Graphs (IDGs) from heterogeneous infrastructure documentation. Multiple open-weight LLMs independently extract candidate entities and dependencies from normalized evidence, after which the framework separates evidence verification, ontology grounding, entity resolution, dependency alignment, validation, fusion, and human review. Evidence support, ontology reconciliation, endpoint resolution, model agreement, and human validation remain distinct states, while provenance and unresolved cases are preserved throughout. The validated IKB is then projected deterministically into the IDG without introducing new LLM-generated knowledge. Evaluation in nine infrastructure projects shows that entity recovery achieves substantially higher recall than complete directed dependency recovery and that canonical endpoint resolution is a major constraint in dependency construction. Cross-model overlap is also much lower for dependencies than for entities, indicating that the models often produce non-overlapping candidate assertions rather than a stable majority consensus. These findings support an auditable evidence-to-IKB-to-IDG process in which uncertainty is preserved and resolved progressively rather than collapsed into a single confidence or voting decision.

Related papers

• CLAIM: Leading Open-domain Active Clarification of Large Language Models with Uncertainty Measurement [50.91045324738942]
  CLAIM is an uncertainty-driven framework for active clarification learning in open-domain settings.<n>We propose an entropy-driven synthetic data generation pipeline that integrates uncertainty estimation with semantic clustering and reasoning-based judgments.<n> Experimental results demonstrate that CLAIM can learn stable and generalizable clarification strategies without relying on manually labeled data.
  arXiv  Detail & Related papers  (2026-08-12T04:27:45Z)
• COSMO: Consensus-Driven Shift Modulation for Source-Free Domain Adaptation [79.43842208099619]
  Source-free domain adaptation (SFDA) adapts a source-trained model to an unlabeled target domain without source data.<n>We formulate VLM-guided SFDA as a sample-wise reliability-allocation problem.<n>We propose Consensus-Driven Shift Modulation (COSMO) to replace expert-to-expert guidance with co-adaptation.
  arXiv  Detail & Related papers  (2026-08-05T09:10:30Z)
• Structural Validation of LLM-Generated Microservice Decompositions Using Source-Code Dependencies [0.2435875827602373]
  Decomposing monolithic systems into prompting is a key activity in software modernization.<n>This paper evaluates the structural adherence of microservice decompositions generated by OpenAI o3 for the PetClinic and Bookstore systems.
  arXiv  Detail & Related papers  (2026-07-30T15:00:21Z)
• HiEviDR-Bench: A Benchmark for Hierarchical Evidence Aggregation in Deep Research [52.87128503345243]
  We introduce HiEviDR-Bench, a benchmark for evaluating Hierarchical Evidence Aggregation in Deep Research.<n>HiEviDR-Bench represents each instance with an explicit evidence graph that captures evidence selection, cross-source linking, and aggregation.<n>We develop a traceability-oriented evaluation framework with five dimensions: report quality, evidence traceability, citation accuracy, claim verification, and answer correctness.
  arXiv  Detail & Related papers  (2026-07-27T23:50:46Z)
• MultAttnAttrib: Training-Free Multimodal Attribution in Long Document Question Answering [73.47440561592556]
  We introduce MultAttnAttrib, a training-free attribution-generation method.<n>We show that MultAttnAttrib consistently outperforms a variety of attribution-generation methods.<n>Our method produces attributions at up to one-seventh of the direct inference latency compared to prompting.
  arXiv  Detail & Related papers  (2026-07-01T19:29:19Z)
• Stateful Evidence-Driven Retrieval-Augmented Generation with Iterative Reasoning [7.701766930531068]
  Stateful Evidence-Driven RAG with Iterative Reasoning is a framework that models question answering as a progressive evidence accumulation process.<n>The framework performs evidence-driven deficiency analysis to identify gaps and conflicts and iteratively refines queries to guide subsequent retrieval.<n> Experiments on multiple question answering benchmarks demonstrate consistent improvements over standard RAG and multi-step baselines.
  arXiv  Detail & Related papers  (2026-03-25T06:57:49Z)
• Leveraging LLM Parametric Knowledge for Fact Checking without Retrieval [60.25608870901428]
  Trustworthiness is a core research challenge for agentic AI systems built on Large Language Models (LLMs)<n>We propose the task of fact-checking without retrieval, focusing on the verification of arbitrary natural language claims, independent of their source robustness.
  arXiv  Detail & Related papers  (2026-03-05T18:42:51Z)
• Diagnosing Structural Failures in LLM-Based Evidence Extraction for Meta-Analysis [0.8193467416247519]
  Review and meta-analyses rely on converting narrative articles into structured, numerically grounded study records.<n>Despite rapid advances in large language models (LLMs), it remains unclear whether they can meet the structural requirements of this process.<n>We propose a structural, diagnostic framework that evaluates LLM-based evidence extraction as a progression of schema-constrained queries.
  arXiv  Detail & Related papers  (2026-02-11T14:09:43Z)
• Hallucination Detection and Mitigation in Large Language Models [0.0]
  Large Language Models (LLMs) and Large Reasoning Models (LRMs) offer transformative potential for high-stakes domains like finance and law.<n>Their tendency to hallucinate, generating factually incorrect or unsupported content, poses a critical reliability risk.<n>This paper introduces a comprehensive framework for hallucination management, built on a continuous improvement cycle driven by root cause awareness.
  arXiv  Detail & Related papers  (2026-01-14T23:19:37Z)
• CompassVerifier: A Unified and Robust Verifier for LLMs Evaluation and Outcome Reward [50.97588334916863]
  We develop CompassVerifier, an accurate and robust lightweight verifier model for evaluation and outcome reward.<n>It demonstrates multi-domain competency spanning math, knowledge, and diverse reasoning tasks, with the capability to process various answer types.<n>We introduce VerifierBench benchmark comprising model outputs collected from multiple data sources, augmented through manual analysis of metaerror patterns to enhance CompassVerifier.
  arXiv  Detail & Related papers  (2025-08-05T17:55:24Z)
• Graph-based RAG Enhancement via Global Query Disambiguation and Dependency-Aware Reranking [9.280502741892676]
  PankRAG is a globally aware, hierarchical query-resolution strategy with a novel dependency-aware reranking mechanism.<n>It applies its dependency-aware reranker to exploit the dependency structure among resolved sub-questions.<n>PankRAG consistently outperforms state-of-the-art approaches across multiple benchmarks.
  arXiv  Detail & Related papers  (2025-06-07T07:17:14Z)
• Retrieval is Not Enough: Enhancing RAG Reasoning through Test-Time Critique and Optimization [58.390885294401066]
  Retrieval-augmented generation (RAG) has become a widely adopted paradigm for enabling knowledge-grounded large language models (LLMs)<n>RAG pipelines often fail to ensure that model reasoning remains consistent with the evidence retrieved, leading to factual inconsistencies or unsupported conclusions.<n>We propose AlignRAG, a novel iterative framework grounded in Critique-Driven Alignment (CDA)<n>We introduce AlignRAG-auto, an autonomous variant that dynamically terminates refinement, removing the need to pre-specify the number of critique iterations.
  arXiv  Detail & Related papers  (2025-04-21T04:56:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.