Explainability Framework for Policy-Aware Autonomous Agents

• URL: http://arxiv.org/abs/2607.21209v1
• Date: Thu, 23 Jul 2026 11:22:25 GMT
• Title: Explainability Framework for Policy-Aware Autonomous Agents
• Abstract summary: We propose a framework that outlines how to produce comprehensible explanations for policy-aware agents.<n>This framework is designed using insights from the social sciences on how to produce good explanations.<n>It is implemented in the Answer Set Programming language while using Python to assist with information extraction and natural-language translation.
• Score: 1.2891210250935148
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In the field of Artificial Intelligence, an agent is a system which is able to autonomously make decisions in order to reach a desired goal. As these systems grow more prevalent in our day-to-day lives, there has been an increased need to add explainability features which can provide an account for an agent's behavior. We therefore propose a framework that outlines how to produce comprehensible explanations for policy-aware agents, or agents which have rule-enforcing policies incorporated in their decision-making framework. This framework is designed using insights from the social sciences on how to produce good explanations. It is implemented in the Answer Set Programming language while using Python to assist with information extraction and natural-language translation. Because these agents incur penalties when violating policies, we are able to leverage these penalties to detect undesirable events in scenarios that are counterfactual to the agents' original actions. This lends itself to creating contrastive explanations (e.g., "the agent performed this action because, had it not, undesirable event X would have occurred."), which formulate the core component for our explainability framework. The framework is evaluated using a survey wherein human participants provide feedback on our program-generated explanations.

Related papers

• Explaining AI Agents Through Execution Traces [1.8862116040247214]
  We present a post-hoc XAI framework that transforms a lengthy agent's execution trace into a structured report and a faithful natural-language explanation.<n>Because it relies solely on execution traces, the framework applies across different agent architectures, environments, and tasks.
  arXiv  Detail & Related papers  (2026-09-05T12:45:56Z)
• SkillJect: Automating Stealthy Skill-Based Prompt Injection for Coding Agents with Trace-Driven Closed-Loop Refinement [120.52289344734415]
  We propose an automated framework for stealthy prompt injection tailored to agent skills.<n>The framework forms a closed loop with three agents: an Attack Agent that synthesizes injection skills under explicit stealth constraints, a Code Agent that executes tasks using the injected skills and an Evaluate Agent that logs action traces.<n>Our method consistently achieves high attack success rates under realistic settings.
  arXiv  Detail & Related papers  (2026-02-15T16:09:48Z)
• The Why Behind the Action: Unveiling Internal Drivers via Agentic Attribution [63.61358761489141]
  Large Language Model (LLM)-based agents are widely used in real-world applications such as customer service, web navigation, and software engineering.<n>We propose a novel framework for textbfgeneral agentic attribution, designed to identify the internal factors driving agent actions regardless of the task outcome.<n>We validate our framework across a diverse suite of agentic scenarios, including standard tool use and subtle reliability risks like memory-induced bias.
  arXiv  Detail & Related papers  (2026-01-21T15:22:21Z)
• Real-Time Reasoning Agents in Evolving Environments [52.21796134114843]
  We introduce real-time reasoning as a new problem formulation for agents in evolving environments.<n>Our work establishes real-time reasoning as a critical testbed for developing practical agents.
  arXiv  Detail & Related papers  (2025-11-07T00:51:02Z)
• Toward a Theory of Agents as Tool-Use Decision-Makers [89.26889709510242]
  We argue that true autonomy requires agents to be grounded in a coherent epistemic framework that governs what they know, what they need to know, and how to acquire that knowledge efficiently.<n>We propose a unified theory that treats internal reasoning and external actions as equivalent epistemic tools, enabling agents to systematically coordinate introspection and interaction.<n>This perspective shifts the design of agents from mere action executors to knowledge-driven intelligence systems, offering a principled path toward building foundation agents capable of adaptive, efficient, and goal-directed behavior.
  arXiv  Detail & Related papers  (2025-06-01T07:52:16Z)
• Intention-aware policy graphs: answering what, how, and why in opaque agents [0.1398098625978622]
  Agents are a special kind of AI-based software in that they interact in complex environments and have increased potential for emergent behaviour. We propose a Probabilistic Graphical Model along with a pipeline for designing such model. We contribute measurements that evaluate the interpretability and reliability of explanations provided. This model can be constructed by taking partial observations of the agent's actions and world states.
  arXiv  Detail & Related papers  (2024-09-27T09:31:45Z)
• DANLI: Deliberative Agent for Following Natural Language Instructions [9.825482203664963]
  We propose a neuro-symbolic deliberative agent that applies reasoning and planning based on its neural and symbolic representations acquired from past experience. We show that our deliberative agent achieves greater than 70% improvement over reactive baselines on the challenging TEACh benchmark.
  arXiv  Detail & Related papers  (2022-10-22T15:57:01Z)
• Explaining Preference-driven Schedules: the EXPRES Framework [23.47188079362728]
  We introduce the EXPRES framework, which can explain why a preference was unsatisfied in a given optimal schedule. The EXPRES framework consists of: (i) an explanation generator that, based on a Mixed-Integer Linear Programming model, finds the best set of reasons that can explain an unsatisfied preference; and (ii) an explanation, which translates the generated explanations into human interpretable ones.
  arXiv  Detail & Related papers  (2022-03-16T19:15:21Z)
• Explaining Reinforcement Learning Policies through Counterfactual Trajectories [147.7246109100945]
  A human developer must validate that an RL agent will perform well at test-time. Our method conveys how the agent performs under distribution shifts by showing the agent's behavior across a wider trajectory distribution. In a user study, we demonstrate that our method enables users to score better than baseline methods on one of two agent validation tasks.
  arXiv  Detail & Related papers  (2022-01-29T00:52:37Z)
• Inherently Explainable Reinforcement Learning in Natural Language [14.117921448623342]
  We focus on the task of creating a reinforcement learning agent that is inherently explainable. This Hierarchically Explainable Reinforcement Learning agent operates in Interactive Fictions, text-based game environments. Our agent is designed to treat explainability as a first-class citizen.
  arXiv  Detail & Related papers  (2021-12-16T14:24:35Z)
• CausalCity: Complex Simulations with Agency for Causal Discovery and Reasoning [68.74447489372037]
  We present a high-fidelity simulation environment that is designed for developing algorithms for causal discovery and counterfactual reasoning. A core component of our work is to introduce textitagency, such that it is simple to define and create complex scenarios. We perform experiments with three state-of-the-art methods to create baselines and highlight the affordances of this environment.
  arXiv  Detail & Related papers  (2021-06-25T00:21:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.