Interactive Explanations for Reinforcement-Learning Agents

• URL: http://arxiv.org/abs/2504.05393v1
• Date: Mon, 07 Apr 2025 18:00:50 GMT
• Title: Interactive Explanations for Reinforcement-Learning Agents
• Authors: Yotam Amitai, Ofra Amir, Guy Avni,
• Abstract summary: We present ASQ-IT -- an interactive explanation system that presents video clips of the agent acting in its environment based on queries given by the user that describe temporal properties of behaviors of interest.<n>Our approach is based on formal methods: queries in ASQ-IT's user interface map to a fragment of Linear Temporal Logic over finite traces (LTLf), which we developed, and our algorithm for query processing is based on automata theory.
• Score: 10.17968794823259
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: As reinforcement learning methods increasingly amass accomplishments, the need for comprehending their solutions becomes more crucial. Most explainable reinforcement learning (XRL) methods generate a static explanation depicting their developers' intuition of what should be explained and how. In contrast, literature from the social sciences proposes that meaningful explanations are structured as a dialog between the explainer and the explainee, suggesting a more active role for the user and her communication with the agent. In this paper, we present ASQ-IT -- an interactive explanation system that presents video clips of the agent acting in its environment based on queries given by the user that describe temporal properties of behaviors of interest. Our approach is based on formal methods: queries in ASQ-IT's user interface map to a fragment of Linear Temporal Logic over finite traces (LTLf), which we developed, and our algorithm for query processing is based on automata theory. User studies show that end-users can understand and formulate queries in ASQ-IT and that using ASQ-IT assists users in identifying faulty agent behaviors.

Related papers

• CLEAR-KGQA: Clarification-Enhanced Ambiguity Resolution for Knowledge Graph Question Answering [13.624962763072899]
  KGQA systems typically assume user queries are unambiguous, which is an assumption that rarely holds in real-world applications. We propose a novel framework that dynamically handles both entity ambiguity (e.g., distinguishing between entities with similar names) and intent ambiguity (e.g., clarifying different interpretations of user queries) through interactive clarification.
  arXiv  Detail & Related papers  (2025-04-13T17:34:35Z)
• Understanding Learner-LLM Chatbot Interactions and the Impact of Prompting Guidelines [9.834055425277874]
  This study investigates learner-AI interactions through an educational experiment in which participants receive structured guidance on effective prompting. To assess user behavior and prompting efficacy, we analyze a dataset of 642 interactions from 107 users. Our findings provide a deeper understanding of how users engage with Large Language Models and the role of structured prompting guidance in enhancing AI-assisted communication.
  arXiv  Detail & Related papers  (2025-04-10T15:20:43Z)
• QLASS: Boosting Language Agent Inference via Q-Guided Stepwise Search [89.97082652805904]
  We propose QLASS (Q-guided Language Agent Stepwise Search), to automatically generate annotations by estimating Q-values.<n>With the stepwise guidance, we propose a Q-guided generation strategy to enable language agents to better adapt to long-term value.<n>We empirically demonstrate that QLASS can lead to more effective decision making through qualitative analysis.
  arXiv  Detail & Related papers  (2025-02-04T18:58:31Z)
• Online inductive learning from answer sets for efficient reinforcement learning exploration [52.03682298194168]
  We exploit inductive learning of answer set programs to learn a set of logical rules representing an explainable approximation of the agent policy.<n>We then perform answer set reasoning on the learned rules to guide the exploration of the learning agent at the next batch.<n>Our methodology produces a significant boost in the discounted return achieved by the agent, even in the first batches of training.
  arXiv  Detail & Related papers  (2025-01-13T16:13:22Z)
• Pangu-Agent: A Fine-Tunable Generalist Agent with Structured Reasoning [50.47568731994238]
  Key method for creating Artificial Intelligence (AI) agents is Reinforcement Learning (RL) This paper presents a general framework model for integrating and learning structured reasoning into AI agents' policies.
  arXiv  Detail & Related papers  (2023-12-22T17:57:57Z)
• FIND: A Function Description Benchmark for Evaluating Interpretability Methods [86.80718559904854]
  This paper introduces FIND (Function INterpretation and Description), a benchmark suite for evaluating automated interpretability methods. FIND contains functions that resemble components of trained neural networks, and accompanying descriptions of the kind we seek to generate. We evaluate methods that use pretrained language models to produce descriptions of function behavior in natural language and code.
  arXiv  Detail & Related papers  (2023-09-07T17:47:26Z)
• AVIS: Autonomous Visual Information Seeking with Large Language Model Agent [123.75169211547149]
  We propose an autonomous information seeking visual question answering framework, AVIS. Our method leverages a Large Language Model (LLM) to dynamically strategize the utilization of external tools. AVIS achieves state-of-the-art results on knowledge-intensive visual question answering benchmarks such as Infoseek and OK-VQA.
  arXiv  Detail & Related papers  (2023-06-13T20:50:22Z)
• Causal Explanations for Sequential Decision-Making in Multi-Agent Systems [31.674391914683888]
  CEMA is a framework for creating causal natural language explanations of an agent's decisions in sequential multi-agent systems. We show CEMA correctly identifies the causes behind the agent's decisions, even when a large number of other agents is present. We show via a user study that CEMA's explanations have a positive effect on participants' trust in autonomous vehicles.
  arXiv  Detail & Related papers  (2023-02-21T16:34:07Z)
• ASQ-IT: Interactive Explanations for Reinforcement-Learning Agents [7.9603223299524535]
  We present ASQ-IT -- an interactive tool that presents video clips of the agent acting in its environment based on queries given by the user that describe temporal properties of behaviors of interest. Our approach is based on formal methods: queries in ASQ-IT's user interface map to a fragment of Linear Temporal Logic over finite traces (LTLf), which we developed, and our algorithm for query processing is based on automata theory.
  arXiv  Detail & Related papers  (2023-01-24T11:57:37Z)
• Semantic Interactive Learning for Text Classification: A Constructive Approach for Contextual Interactions [0.0]
  We propose a novel interaction framework called Semantic Interactive Learning for the text domain. We frame the problem of incorporating constructive and contextual feedback into the learner as a task to find an architecture that enables more semantic alignment between humans and machines. We introduce a technique called SemanticPush that is effective for translating conceptual corrections of humans to non-extrapolating training examples.
  arXiv  Detail & Related papers  (2022-09-07T08:13:45Z)
• Generating User-Centred Explanations via Illocutionary Question Answering: From Philosophy to Interfaces [3.04585143845864]
  We show a new approach for the generation of interactive explanations based on a sophisticated pipeline of AI algorithms. Our contribution is an approach to frame illocution in a computer-friendly way, to achieve user-centrality with statistical question answering. We tested our hypotheses with a user-study involving more than 60 participants, on two XAI-based systems.
  arXiv  Detail & Related papers  (2021-10-02T09:06:36Z)
• From Philosophy to Interfaces: an Explanatory Method and a Tool Inspired by Achinstein's Theory of Explanation [3.04585143845864]
  We propose a new method for explanations in Artificial Intelligence (AI) We show a new approach for the generation of interactive explanations based on a pipeline of AI algorithms. We tested our hypothesis on a well-known XAI-powered credit approval system by IBM.
  arXiv  Detail & Related papers  (2021-09-09T11:10:03Z)
• A general framework for scientifically inspired explanations in AI [76.48625630211943]
  We instantiate the concept of structure of scientific explanation as the theoretical underpinning for a general framework in which explanations for AI systems can be implemented. This framework aims to provide the tools to build a "mental-model" of any AI system so that the interaction with the user can provide information on demand and be closer to the nature of human-made explanations.
  arXiv  Detail & Related papers  (2020-03-02T10:32:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.