An Agentic RAG and Evaluation Framework for Assurance Case Generation: Industrial Use Case for the EU Cyber Resilience Act Compliance

• URL: http://arxiv.org/abs/2608.19509v1
• Date: Wed, 19 Aug 2026 23:54:30 GMT
• Title: An Agentic RAG and Evaluation Framework for Assurance Case Generation: Industrial Use Case for the EU Cyber Resilience Act Compliance
• Abstract summary: Complying with the EU Cyber Resilience Act (CRA) is a resource-intensive challenge for SMEs.<n>We introduce an automated framework for generating Assurance Cases (ACs) using an agentic Retrieval-Augmented Generation.<n>By systematically mapping technical documentation requirements, the framework streamlines the generation of certification evidence.
• Score: 1.3121550601837717
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Complying with the EU Cyber Resilience Act (CRA) is a resource-intensive challenge for SMEs due to the complexity of cybersecurity conformity assessment. Yet, it is essential for demonstrating regulatory compliance and ensuring product security and resilience. To address this, we introduce an automated framework for generating Assurance Cases (ACs) using an agentic Retrieval-Augmented Generation grounded in a formal Claim-Argument-Evidence logic. By systematically mapping technical documentation requirements, the framework streamlines the generation of certification evidence. We validate our approach on a case study of Catalink's PATROLIoT wildfire monitoring system, where the agentic RAG generated 70 ACs with high grounding density (~4.4 artefacts per AC). The proposed Natural Language Inference (NLI) evaluator achieves 0.88 accuracy, which provides robust evidence grounding and traceability, while expert-validated plausibility (3.06) supports interpretable justifications. For practitioners, this work provides a scalable, interpretable approach for automating mandatory CRA conformity assessments, reducing manual effort while maintaining transparent decision support.

Related papers

• AI Slop and Hallucinations in Vulnerability Assessment: A Survey on Reasoning Failures and Trustworthy Mitigation [48.55515767840701]
  "AI slop" is artifacts, hallucinated vulnerabilities, plausible but incorrect patches, and semantically repackaged bug reports.<n>This paper surveys the empirical evidence, identifies a unifying mechanism, and traces a path toward trustworthy triage.
  arXiv  Detail & Related papers  (2026-08-26T11:48:38Z)
• Beyond Goodhart's Law: A Dynamic Benchmark for Evaluating Compliance in Multi-Agent Systems [45.89982596338141]
  We introduce MAC-Bench, a dynamic, adversarial benchmark designed to evaluate the procedural alignment of multi-agent systems under realistic pressure.<n>We introduce novel metrics: the Compliance-Weighted Success Rate (CSR) and the Machiavellian Gap (MG)<n>We conduct a comprehensive evaluation of state-of-the-art frontier models to reveal the pervasive trade-offs between success and compliance.
  arXiv  Detail & Related papers  (2026-06-05T19:33:58Z)
• A New Framework for Cybersecurity Refusals in AI Agents [52.94784168139071]
  We present the first framework for establishing refusal boundaries in offensive security contexts.<n>We apply this framework to assess how current LLM-powered agents adhere to appropriate refusal boundaries across a range of web-based offensive security scenarios.
  arXiv  Detail & Related papers  (2026-05-31T15:39:39Z)
• Bounding the Black Box: A Statistical Certification Framework for AI Risk Regulation [0.0]
  Governments have responded: the EU AI Act, the NIST Risk Management Framework, and the Council of Europe Convention all demand that high-risk systems demonstrate safety before deployment.<n>Beneath this regulatory consensus lies a critical vacuum: none specifies what acceptable risk'' means in quantitative terms.<n>We propose a two-stage framework that transforms AI risk regulation into engineering practice.
  arXiv  Detail & Related papers  (2026-04-23T16:50:35Z)
• Compliance-by-Construction Argument Graphs: Using Generative AI to Produce Evidence-Linked Formal Arguments for Certification-Grade Accountability [0.0]
  This paper proposes a compliance-by-construction architecture that integrates Generative AI (GenAI) with structured formal argument representations.<n>The architecture treats each AI-assisted step as a claim that must be supported by verifiable evidence and validated against explicit reasoning constraints.<n>The analysis suggests that validation rules can prevent unsupported claims from entering the decision record while allowing GenAI to accelerate argument construction.
  arXiv  Detail & Related papers  (2026-04-05T12:55:16Z)
• Beyond Benchmark Islands: Toward Representative Trustworthiness Evaluation for Agentic AI [45.21562889170875]
  We propose a systematic evaluation paradigm that characterizes agent trustworthiness over a scenario manifold.<n>The framework integrates four complementary components: static cognitive and policy analysis, (ii) interactive sandbox simulation, (iii) social-ethical alignment assessment, and (iv) a distribution-aware representative sampling engine.
  arXiv  Detail & Related papers  (2026-03-16T08:51:33Z)
• Towards Verifiably Safe Tool Use for LLM Agents [53.55621104327779]
  Large language model (LLM)-based AI agents extend capabilities by enabling access to tools such as data sources, APIs, search engines, code sandboxes, and even other agents.<n>LLMs may invoke unintended tool interactions and introduce risks, such as leaking sensitive data or overwriting critical records.<n>Current approaches to mitigate these risks, such as model-based safeguards, enhance agents' reliability but cannot guarantee system safety.
  arXiv  Detail & Related papers  (2026-01-12T21:31:38Z)
• AI Deployment Authorisation: A Global Standard for Machine-Readable Governance of High-Risk Artificial Intelligence [0.0]
  This paper introduces the AI Deployment Authorisation Score (ADAS), a machine-readable regulatory framework that evaluates AI systems.<n>ADAS produces a cryptographically verifiable deployment certificate that regulators, insurers, and infrastructure operators can consume as a license to operate.
  arXiv  Detail & Related papers  (2026-01-11T18:14:20Z)
• SafeEvalAgent: Toward Agentic and Self-Evolving Safety Evaluation of LLMs [37.82193156438782]
  This paper introduces a new paradigm of agentic safety evaluation, reframing evaluation as a continuous and self-evolving process.<n>We propose a novel multi-agent framework SafeEvalAgent, which autonomously ingests unstructured policy documents to generate and perpetually evolve a comprehensive safety benchmark.<n>Our experiments demonstrate the effectiveness of SafeEvalAgent, showing a consistent decline in model safety as the evaluation hardens.
  arXiv  Detail & Related papers  (2025-09-30T11:20:41Z)
• Safe and Certifiable AI Systems: Concepts, Challenges, and Lessons Learned [45.44933002008943]
  This white paper presents the T"UV AUSTRIA Trusted AI framework.<n>It is an end-to-end audit catalog and methodology for assessing and certifying machine learning systems.<n>Building on three pillars - Secure Software Development, Functional Requirements, and Ethics & Data Privacy - it translates the high-level obligations of the EU AI Act into specific, testable criteria.
  arXiv  Detail & Related papers  (2025-09-08T17:52:08Z)
• AI-Supported Platform for System Monitoring and Decision-Making in Nuclear Waste Management with Large Language Models [1.6795461001108096]
  This paper presents a multi-agent Retrieval-Augmented Generation (RAG) system that integrates large language models (LLMs) with document retrieval mechanisms.<n>The system ensures regulatory decisions remain factually grounded, dynamically adapting to evolving regulatory frameworks.
  arXiv  Detail & Related papers  (2025-05-27T20:29:53Z)
• Multi-Stage Retrieval for Operational Technology Cybersecurity Compliance Using Large Language Models: A Railway Casestudy [1.1010026679581653]
  This paper proposes a novel system that leverages Large Language Models (LLMs) and multi-stage retrieval to enhance the compliance verification process.<n>We first evaluate a Baseline Compliance Architecture (BCA) for answering OTCS compliance queries, then develop an extended approach called Parallel Compliance Architecture (PCA)<n>We demonstrate that the PCA significantly improves both correctness and reasoning quality in compliance verification.
  arXiv  Detail & Related papers  (2025-04-18T19:24:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.