Tool for Supporting Debugging and Understanding of Normative Requirements Using LLMs

• URL: http://arxiv.org/abs/2507.05504v1
• Date: Mon, 07 Jul 2025 21:57:28 GMT
• Title: Tool for Supporting Debugging and Understanding of Normative Requirements Using LLMs
• Abstract summary: Normative requirements specify social, legal, ethical, empathetic, and cultural (SLEEC) norms that must be observed by a system.<n>These requirements are typically defined by stakeholders in the non-technical system with diverse expertise.<n>SLEEC-LLM improves the efficiency and explainability of normative requirements elicitation and consistency analysis.
• Score: 3.7885668021375465
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Normative requirements specify social, legal, ethical, empathetic, and cultural (SLEEC) norms that must be observed by a system. To support the identification of SLEEC requirements, numerous standards and regulations have been developed. These requirements are typically defined by stakeholders in the non-technical system with diverse expertise (e.g., ethicists, lawyers, social scientists). Hence, ensuring their consistency and managing the requirement elicitation process are complex and error-prone tasks. Recent research has addressed this challenge using domain-specific languages to specify normative requirements as rules, whose consistency can then be analyzed with formal methods. Nevertheless, these approaches often present the results from formal verification tools in a way that is inaccessible to non-technical users. This hinders understanding and makes the iterative process of eliciting and validating these requirements inefficient in terms of both time and effort. To address this problem, we introduce SLEEC-LLM, a tool that uses large language models (LLMs) to provide natural-language interpretations for model-checking counterexamples corresponding to SLEEC rule inconsistencies. SLEEC-LLM improves the efficiency and explainability of normative requirements elicitation and consistency analysis. To demonstrate its effectiveness, we summarise its use in two real-world case studies involving non-technical stakeholders.

Related papers

• Grounded Normative Rule Generation with Structured Search [47.28595196084396]
  We formalize the problem as Grounded Normative Rule Synthesis (GNRS)<n>We introduce GNRS-Search, a framework that utilizes Markov Chain Monte Carlo (MCMC) sampling to optimize a discrete, five-slot And-Or Graph (AOG)<n>We evaluate our approach on GNRS-Bench, a benchmark spanning 116 controlled goals across eight scene families, and RealCharterBench, which evaluates transfer to 53 real-derived policy tasks with hidden source clauses.
  arXiv  Detail & Related papers  (2026-08-23T05:43:42Z)
• CLAIM: Leading Open-domain Active Clarification of Large Language Models with Uncertainty Measurement [50.91045324738942]
  CLAIM is an uncertainty-driven framework for active clarification learning in open-domain settings.<n>We propose an entropy-driven synthetic data generation pipeline that integrates uncertainty estimation with semantic clustering and reasoning-based judgments.<n> Experimental results demonstrate that CLAIM can learn stable and generalizable clarification strategies without relying on manually labeled data.
  arXiv  Detail & Related papers  (2026-08-12T04:27:45Z)
• Automatic Translation of Unstructured Requirements into Linear Temporal Logic through Large Language Models [46.28810845443601]
  This paper evaluates whether contemporary off-the-shelf Large Language Models (LLMs) can help bridge this gap by generating Linear Temporal Logic (LTL) formulas directly from unstructured requirements.<n>The study examines six modern LLMs using a few-shot prompting strategy on a heterogeneous benchmark of 15 structurally varied requirements.<n>The results indicate that current general-purpose LLMs can achieve practically significant performance on the unstructured NLto-LTL task without task-specific fine-tuning.
  arXiv  Detail & Related papers  (2026-08-06T17:13:07Z)
• Learning and Enforcing Context-Sensitive Control for LLMs [48.58004364546273]
  We introduce a framework that automatically learns context-sensitive constraints from Large Language Models interactions.<n>Experiments demonstrate that our method enables even small LLMs (1B parameters) to learn and generate with perfect constraint adherence.
  arXiv  Detail & Related papers  (2026-04-12T14:50:03Z)
• Social, Legal, Ethical, Empathetic and Cultural Norm Operationalisation for AI Agents [5.282637986114336]
  AI agents are increasingly used in high-stakes domains like healthcare and law enforcement.<n> aligning their behaviour with social, legal, ethical, empathetic, and cultural (SLEEC) norms has become a critical engineering challenge.<n>We propose a systematic SLEEC-norm operationalisation process for determining, validating, implementing, and verifying normative requirements.
  arXiv  Detail & Related papers  (2026-03-12T12:37:39Z)
• The Necessity of a Unified Framework for LLM-Based Agent Evaluation [46.631678638677386]
  General-purpose agents have seen fundamental advancements.<n> evaluating these agents presents unique challenges that distinguish them from static QA benchmarks.<n>We propose that a unified evaluation framework is essential for the rigorous advancement of agent evaluation.
  arXiv  Detail & Related papers  (2026-02-03T08:18:37Z)
• Judging by the Rules: Compliance-Aligned Framework for Modern Slavery Statement Monitoring [24.13989765643719]
  Modern slavery affects millions of people worldwide, and regulatory frameworks such as Modern Slavery Acts now require companies to publish detailed disclosures.<n>These statements are often vague and inconsistent, making manual review time-consuming and difficult to scale.<n>We propose a novel framework that harnesses AI for rule-level compliance verification while preserving expert oversight.
  arXiv  Detail & Related papers  (2025-11-11T03:41:44Z)
• Safe and Certifiable AI Systems: Concepts, Challenges, and Lessons Learned [45.44933002008943]
  This white paper presents the T"UV AUSTRIA Trusted AI framework.<n>It is an end-to-end audit catalog and methodology for assessing and certifying machine learning systems.<n>Building on three pillars - Secure Software Development, Functional Requirements, and Ethics & Data Privacy - it translates the high-level obligations of the EU AI Act into specific, testable criteria.
  arXiv  Detail & Related papers  (2025-09-08T17:52:08Z)
• Data Dependency-Aware Code Generation from Enhanced UML Sequence Diagrams [54.528185120850274]
  We propose a novel step-by-step code generation framework named API2Dep.<n>First, we introduce an enhanced Unified Modeling Language (UML) API diagram tailored for service-oriented architectures.<n>Second, recognizing the critical role of data flow, we introduce a dedicated data dependency inference task.
  arXiv  Detail & Related papers  (2025-08-05T12:28:23Z)
• Leveraging LLMs for Formal Software Requirements -- Challenges and Prospects [0.0]
  VERIFAI1 aims to investigate automated and semi-automated approaches to bridge this gap.<n>This position paper presents a preliminary synthesis of relevant literature to identify recurring challenges and prospective research directions.
  arXiv  Detail & Related papers  (2025-07-18T19:15:50Z)
• Evaluating Large Language Models for Real-World Engineering Tasks [75.97299249823972]
  This paper introduces a curated database comprising over 100 questions derived from authentic, production-oriented engineering scenarios.<n>Using this dataset, we evaluate four state-of-the-art Large Language Models (LLMs)<n>Our results show that LLMs demonstrate strengths in basic temporal and structural reasoning but struggle significantly with abstract reasoning, formal modeling, and context-sensitive engineering logic.
  arXiv  Detail & Related papers  (2025-05-12T14:05:23Z)
• LEGOS-SLEEC: Tool for Formalizing and Analyzing Normative Requirements [1.4272256806865102]
  We present LEGOS-SLEEC, a tool designed to support interdisciplinary stakeholders in specifying normative requirements.<n>SLEEC DSL has been developed to formalize these requirements as SLEEC rules.<n>We have significantly improved the user interface of LEGOS-SLEEC and its diagnostic support.
  arXiv  Detail & Related papers  (2025-01-21T23:31:04Z)
• The Synergy of LLMs & RL Unlocks Offline Learning of Generalizable Language-Conditioned Policies with Low-fidelity Data [50.544186914115045]
  TEDUO is a novel training pipeline for offline language-conditioned policy learning in symbolic environments.<n>Our approach harnesses large language models (LLMs) in a dual capacity: first, as automatization tools augmenting offline datasets with richer annotations, and second, as generalizable instruction-following agents.
  arXiv  Detail & Related papers  (2024-12-09T18:43:56Z)
• RIRAG: Regulatory Information Retrieval and Answer Generation [51.998738311700095]
  We introduce a task of generating question-passages pairs, where questions are automatically created and paired with relevant regulatory passages.<n>We create the ObliQA dataset, containing 27,869 questions derived from the collection of Abu Dhabi Global Markets (ADGM) financial regulation documents.<n>We design a baseline Regulatory Information Retrieval and Answer Generation (RIRAG) system and evaluate it with RePASs, a novel evaluation metric.
  arXiv  Detail & Related papers  (2024-09-09T14:44:19Z)
• Can Long-Context Language Models Subsume Retrieval, RAG, SQL, and More? [54.667202878390526]
  Long-context language models (LCLMs) have the potential to revolutionize our approach to tasks traditionally reliant on external tools like retrieval systems or databases. We introduce LOFT, a benchmark of real-world tasks requiring context up to millions of tokens designed to evaluate LCLMs' performance on in-context retrieval and reasoning. Our findings reveal LCLMs' surprising ability to rival state-of-the-art retrieval and RAG systems, despite never having been explicitly trained for these tasks.
  arXiv  Detail & Related papers  (2024-06-19T00:28:58Z)
• The Impossibility of Fair LLMs [17.812295963158714]
  We analyze a variety of technical fairness frameworks and find inherent challenges in each that make the development of a fair language model intractable.<n>We show that each framework either does not extend to the general-purpose AI context or is infeasible in practice.<n>These inherent challenges would persist for general-purpose AI, including LLMs, even if empirical challenges, such as limited participatory input and limited measurement methods, were overcome.
  arXiv  Detail & Related papers  (2024-05-28T04:36:15Z)
• Logical analysis and contradiction detection in high-level requirements during the review process using sat-solver [0.0]
  This study introduces a method for analyzing and identifying inconsistencies between high-level requirements using a data dictionary. The goal of this approach is to significantly reduce the review time of high-level requirements in the software verification process.
  arXiv  Detail & Related papers  (2024-04-30T19:26:54Z)
• Normative Requirements Operationalization with Large Language Models [3.456725053685842]
  Normative non-functional requirements specify constraints that a system must observe in order to avoid violations of social, legal, ethical, empathetic, and cultural norms. Recent research has tackled this challenge using a domain-specific language to specify normative requirements. We propose a complementary approach that uses Large Language Models to extract semantic relationships between abstract representations of system capabilities.
  arXiv  Detail & Related papers  (2024-04-18T17:01:34Z)
• Towards an Enforceable GDPR Specification [49.1574468325115]
  Privacy by Design (PbD) is prescribed by modern privacy regulations such as the EU's. One emerging technique to realize PbD is enforcement (RE) We present a set of requirements and an iterative methodology for creating formal specifications of legal provisions.
  arXiv  Detail & Related papers  (2024-02-27T09:38:51Z)
• Large Language Models and Explainable Law: a Hybrid Methodology [44.99833362998488]
  The paper advocates for LLMs to enhance the accessibility, usage and explainability of rule-based legal systems. A methodology is developed to explore the potential use of LLMs for translating the explanations produced by rule-based systems.
  arXiv  Detail & Related papers  (2023-11-20T14:47:20Z)
• From Chaos to Clarity: Claim Normalization to Empower Fact-Checking [57.024192702939736]
  Claim Normalization (aka ClaimNorm) aims to decompose complex and noisy social media posts into more straightforward and understandable forms. We propose CACN, a pioneering approach that leverages chain-of-thought and claim check-worthiness estimation. Our experiments demonstrate that CACN outperforms several baselines across various evaluation measures.
  arXiv  Detail & Related papers  (2023-10-22T16:07:06Z)
• Specification, Validation and Verification of Social, Legal, Ethical, Empathetic and Cultural Requirements for Autonomous Agents [4.673587416936401]
  We introduce a framework for formal specification, validation and verification of social, legal, ethical, empathetic and cultural (SLEEC) rules for autonomous agents. We show the applicability of our framework for two autonomous agents from different domains: a firefighter UAV, and an assistive-dressing robot.
  arXiv  Detail & Related papers  (2023-07-07T16:13:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.