Executable Governance for AI: Translating Policies into Rules Using LLMs

• URL: http://arxiv.org/abs/2512.04408v1
• Date: Thu, 04 Dec 2025 03:11:54 GMT
• Title: Executable Governance for AI: Translating Policies into Rules Using LLMs
• Abstract summary: Policy-to-Tests (P2T) is a framework that converts natural policy documents into normalized, machine-readable rules.<n>To test the framework beyond a single policy, we apply it across general frameworks, sector guidance, and enterprise standards.<n>These AI-generated rules closely match strong human baselines on span-level and rule-level metrics, with robust inter-annotator agreement on the gold set.
• Score: 1.388831902854619
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: AI policy guidance is predominantly written as prose, which practitioners must first convert into executable rules before frameworks can evaluate or enforce them. This manual step is slow, error-prone, difficult to scale, and often delays the use of safeguards in real-world deployments. To address this gap, we present Policy-to-Tests (P2T), a framework that converts natural-language policy documents into normalized, machine-readable rules. The framework comprises a pipeline and a compact domain-specific language (DSL) that encodes hazards, scope, conditions, exceptions, and required evidence, yielding a canonical representation of extracted rules. To test the framework beyond a single policy, we apply it across general frameworks, sector guidance, and enterprise standards, extracting obligation-bearing clauses and converting them into executable rules. These AI-generated rules closely match strong human baselines on span-level and rule-level metrics, with robust inter-annotator agreement on the gold set. To evaluate downstream behavioral and safety impact, we add HIPAA-derived safeguards to a generative agent and compare it with an otherwise identical agent without guardrails. An LLM-based judge, aligned with gold-standard criteria, measures violation rates and robustness to obfuscated and compositional prompts. Detailed results are provided in the appendix. We release the codebase, DSL, prompts, and rule sets as open-source resources to enable reproducible evaluation.

Related papers

• PolicyMem: Geometric Policy Memory for LLM Governance [72.04962480384322]
  We introduce PolicyMem, a geometric policy memory that externalizes natural-language policies as reusable geometric memory objects.<n>A memory writer compiles natural-language policies into policy memory slots, and query-response pairs read the policy memory through projection energy.<n>The resulting policy-evidence profile directly mediates the safety verdict and is reused for policy attribution and post-intervention verification.
  arXiv  Detail & Related papers  (2026-09-12T06:07:43Z)
• Grounded Normative Rule Generation with Structured Search [47.28595196084396]
  We formalize the problem as Grounded Normative Rule Synthesis (GNRS)<n>We introduce GNRS-Search, a framework that utilizes Markov Chain Monte Carlo (MCMC) sampling to optimize a discrete, five-slot And-Or Graph (AOG)<n>We evaluate our approach on GNRS-Bench, a benchmark spanning 116 controlled goals across eight scene families, and RealCharterBench, which evaluates transfer to 53 real-derived policy tasks with hidden source clauses.
  arXiv  Detail & Related papers  (2026-08-23T05:43:42Z)
• Autoformalization of Agent Instructions into Policy-as-Code [0.0]
  We present an autoformalization pipeline that translates agent prompts, MCP tool descriptions, and natural language policy documents into formally verified policies.<n>The resulting policies are written in the Cedar Policy Language.<n>On the MedAgentBench benchmark, our autoformalized policies cover substantially more of the source natural-language specification than the hand-coded symbolic enforcement.
  arXiv  Detail & Related papers  (2026-06-25T06:23:15Z)
• Deontic Policies for Runtime Governance of Agentic AI Systems [2.1095754823099657]
  We propose AgenticRei, which realizes key governance requirements such as obligations, dispensations, policy conflict resolutions, and reasoning over policies.<n>Our approach composes naturally with industry-standard frameworks like A2AS.
  arXiv  Detail & Related papers  (2026-06-17T18:02:07Z)
• Citation-Closure Retrieval and Per-Rule Attribution for Real-World Regulatory Compliance Question Answering [54.97384993676565]
  We formalize Regulatory Compliance QA with RegOps-Bench, a novel benchmark featuring an Operational Knowledge Graph derived from complex national R&D regulations.<n>RefWalk traverses cross-document citations, fuses multi-view candidates via max-based aggregation, and enforces per-rule attribution to explicitly map claims to sources.
  arXiv  Detail & Related papers  (2026-05-28T10:38:38Z)
• Inverting the Shield: Systematically Generating Safety Tests from Policy Specifications [34.429470292997145]
  We introduce a novel framework that brings the rigor of specification-based software testing to AI safety.<n>POLARIS first compiles unstructured natural-language policies into First-Order Logic representations, establishing a traceable link between high-level rules and concrete test cases.<n>This formalization enables the construction of a Semantic Policy Graph, where complex policy violation scenarios are encoded as traversable paths.<n>Experiments demonstrate that POLARIS achieves higher policy coverage and attack success counts compared to established baselines.
  arXiv  Detail & Related papers  (2026-05-24T05:51:23Z)
• PolicyBank: Evolving Policy Understanding for LLM Agents [51.86716874651299]
  PolicyBank maintains structured, tool-level policy insights and iteratively refines them.<n>PolicyBank closes up to 82% of the gap toward a human oracle.
  arXiv  Detail & Related papers  (2026-04-16T20:29:30Z)
• ContextLens: Modeling Imperfect Privacy and Safety Context for Legal Compliance [49.524070843587594]
  We propose ContextLens, a semi-rule-based framework that leverages large language models (LLMs) to ground input context in the legal domain.<n>We conduct experiments on existing compliance benchmarks that cover the General Data Protection Regulation () and the EU AI Act.<n>The results suggest that our ContextLens can significantly improve LLMs' compliance assessment and surpass existing baselines without any training.
  arXiv  Detail & Related papers  (2026-04-14T05:35:38Z)
• Training-Free Policy Violation Detection via Activation-Space Whitening in LLMs [21.5603664964501]
  We propose a training-free and efficient method that treats policy violation detection as an out-of-distribution detection problem.<n>Inspired by whitening techniques, we apply a linear transformation to decorrelate the model's hidden activations and standardize them to zero mean and unit variance.<n>On a challenging policy benchmark, our approach achieves state-of-the-art results, surpassing both existing guardrails and fine-tuned reasoning models.
  arXiv  Detail & Related papers  (2025-12-03T17:23:39Z)
• RulePilot: An LLM-Powered Agent for Security Rule Generation [15.638534363355342]
  Rule-based detection often identifies malicious logs based on the predefined grammar logic.<n>In this paper, we propose RulePilot, which mimics human expertise via LLM-based agent.<n>Using RulePilot, the security analysts do not need to write down the rules following the grammar, instead, they can just provide the annotations.
  arXiv  Detail & Related papers  (2025-11-15T13:59:16Z)
• Analyzing and Internalizing Complex Policy Documents for LLM Agents [53.14898416858099]
  Large Language Model (LLM)-based agentic systems rely on in-context policy documents encoding diverse business rules.<n>This motivates developing internalization methods that embed policy documents into model priors while preserving performance.<n>We introduce CC-Gen, an agentic benchmark generator with Controllable Complexity across four levels.
  arXiv  Detail & Related papers  (2025-10-13T16:30:07Z)
• Policy-as-Prompt: Turning AI Governance Rules into Guardrails for AI Agents [0.19336815376402716]
  We introduce a regulatory machine learning framework that converts unstructured design artifacts (like PRDs, TDDs, and code) into verifiable runtime guardrails.<n>Our Policy as Prompt method reads these documents and risk controls to build a source-linked policy tree.<n>System is built to enforce least privilege and data minimization.
  arXiv  Detail & Related papers  (2025-09-28T17:36:52Z)
• Scaling Policy Compliance Assessment in Language Models with Policy Reasoning Traces [12.671657542087624]
  Policy Reasoning Traces (PRT) is a form of specialized generated reasoning chains that serve as a reasoning bridge to improve an LLM's policy compliance assessment capabilities.<n>Our empirical evaluations demonstrate that the use of PRTs for both inference-time and training-time scenarios significantly enhances the performance of open-weight and commercial models.
  arXiv  Detail & Related papers  (2025-09-27T13:10:21Z)
• Rethinking Testing for LLM Applications: Characteristics, Challenges, and a Lightweight Interaction Protocol [83.83217247686402]
  Large Language Models (LLMs) have evolved from simple text generators into complex software systems that integrate retrieval augmentation, tool invocation, and multi-turn interactions.<n>Their inherent non-determinism, dynamism, and context dependence pose fundamental challenges for quality assurance.<n>This paper decomposes LLM applications into a three-layer architecture: textbftextitSystem Shell Layer, textbftextitPrompt Orchestration Layer, and textbftextitLLM Inference Core.
  arXiv  Detail & Related papers  (2025-08-28T13:00:28Z)
• Few-shot Policy (de)composition in Conversational Question Answering [54.259440408606515]
  We propose a neuro-symbolic framework to detect policy compliance using large language models (LLMs) in a few-shot setting.<n>We show that our approach soundly reasons about policy compliance conversations by extracting sub-questions to be answered, assigning truth values from contextual information, and explicitly producing a set of logic statements from the given policies.<n>We apply this approach to the popular PCD and conversational machine reading benchmark, ShARC, and show competitive performance with no task-specific finetuning.
  arXiv  Detail & Related papers  (2025-01-20T08:40:15Z)
• DECIDER: A Dual-System Rule-Controllable Decoding Framework for Language Generation [57.07295906718989]
  Constrained decoding approaches aim to control the meaning or style of text generated by pre-trained large language (Ms also PLMs) for various tasks at inference time.<n>These methods often guide plausible continuations by greedily and explicitly selecting targets.<n>Inspired by cognitive dual-process theory, we propose a novel decoding framework DECIDER.
  arXiv  Detail & Related papers  (2024-03-04T11:49:08Z)
• SoFA: Shielded On-the-fly Alignment via Priority Rule Following [90.32819418613407]
  This paper introduces a novel alignment paradigm, priority rule following, which defines rules as the primary control mechanism in each dialog. We present PriorityDistill, a semi-automated approach for distilling priority following signals from simulations to ensure robust rule integration and adherence.
  arXiv  Detail & Related papers  (2024-02-27T09:52:27Z)
• Towards an Enforceable GDPR Specification [49.1574468325115]
  Privacy by Design (PbD) is prescribed by modern privacy regulations such as the EU's. One emerging technique to realize PbD is enforcement (RE) We present a set of requirements and an iterative methodology for creating formal specifications of legal provisions.
  arXiv  Detail & Related papers  (2024-02-27T09:38:51Z)
• Building Rule Hierarchies for Efficient Logical Rule Learning from Knowledge Graphs [20.251630903853016]
  We propose new methods for pruning unpromising rules using rule hierarchies. We show that the application of HPMs is effective in removing unpromising rules.
  arXiv  Detail & Related papers  (2020-06-29T16:33:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.