Interrogating the Black Box: Transparency through Information-Seeking Dialogues

• URL: http://arxiv.org/abs/2102.04714v1
• Date: Tue, 9 Feb 2021 09:14:04 GMT
• Title: Interrogating the Black Box: Transparency through Information-Seeking Dialogues
• Authors: Andrea Aler Tubella, Andreas Theodorou and Juan Carlos Nieves
• Abstract summary: We propose to construct an investigator agent to query a learning agent to investigate its adherence to an ethical policy. This formal dialogue framework is the main contribution of this paper. We argue that the introduced formal dialogue framework opens many avenues both in the area of compliance checking and in the analysis of properties of opaque systems.
• Score: 9.281671380673306
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper is preoccupied with the following question: given a (possibly opaque) learning system, how can we understand whether its behaviour adheres to governance constraints? The answer can be quite simple: we just need to "ask" the system about it. We propose to construct an investigator agent to query a learning agent -- the suspect agent -- to investigate its adherence to a given ethical policy in the context of an information-seeking dialogue, modeled in formal argumentation settings. This formal dialogue framework is the main contribution of this paper. Through it, we break down compliance checking mechanisms into three modular components, each of which can be tailored to various needs in a vast amount of ways: an investigator agent, a suspect agent, and an acceptance protocol determining whether the responses of the suspect agent comply with the policy. This acceptance protocol presents a fundamentally different approach to aggregation: rather than using quantitative methods to deal with the non-determinism of a learning system, we leverage the use of argumentation semantics to investigate the notion of properties holding consistently. Overall, we argue that the introduced formal dialogue framework opens many avenues both in the area of compliance checking and in the analysis of properties of opaque systems.

Related papers

• Few-shot Policy (de)composition in Conversational Question Answering [54.259440408606515]
  We propose a neuro-symbolic framework to detect policy compliance using large language models (LLMs) in a few-shot setting. We show that our approach soundly reasons about policy compliance conversations by extracting sub-questions to be answered, assigning truth values from contextual information, and explicitly producing a set of logic statements from the given policies. We apply this approach to the popular PCD and conversational machine reading benchmark, ShARC, and show competitive performance with no task-specific finetuning.
  arXiv  Detail & Related papers  (2025-01-20T08:40:15Z)
• Who Speaks Next? Multi-party AI Discussion Leveraging the Systematics of Turn-taking in Murder Mystery Games [7.946510318969307]
  This study focuses on conversational norms such as adjacency pairs and turn-taking found in conversation analysis. We propose a new framework called "Murder Mystery Agents" that applies these norms to AI agents' dialogue control. The proposed framework integrates next speaker selection based on adjacency pairs and a self-selection mechanism.
  arXiv  Detail & Related papers  (2024-12-06T10:45:54Z)
• Understanding Understanding: A Pragmatic Framework Motivated by Large Language Models [13.279760256875127]
  In Turing-test fashion, the framework is based solely on the agent's performance, and specifically on how well it answers questions. We show how high confidence can be achieved via random sampling and the application of probabilistic confidence bounds.
  arXiv  Detail & Related papers  (2024-06-16T13:37:08Z)
• Leveraging Counterfactual Paths for Contrastive Explanations of POMDP Policies [2.4332936182093197]
  XAI aims to reduce confusion and foster trust in systems by providing explanations of agent behavior. POMDPs provide a flexible framework capable of reasoning over transition and state uncertainty. This work investigates the use of user-provided counterfactuals to generate contrastive explanations of POMDP policies.
  arXiv  Detail & Related papers  (2024-03-28T18:19:38Z)
• Clarify When Necessary: Resolving Ambiguity Through Interaction with LMs [58.620269228776294]
  We propose a task-agnostic framework for resolving ambiguity by asking users clarifying questions. We evaluate systems across three NLP applications: question answering, machine translation and natural language inference. We find that intent-sim is robust, demonstrating improvements across a wide range of NLP tasks and LMs.
  arXiv  Detail & Related papers  (2023-11-16T00:18:50Z)
• A Unifying Framework for Learning Argumentation Semantics [50.69905074548764]
  We present a novel framework, which uses an Inductive Logic Programming approach to learn the acceptability semantics for several abstract and structured argumentation frameworks in an interpretable way. Our framework outperforms existing argumentation solvers, thus opening up new future research directions in the area of formal argumentation and human-machine dialogues.
  arXiv  Detail & Related papers  (2023-10-18T20:18:05Z)
• HOP, UNION, GENERATE: Explainable Multi-hop Reasoning without Rationale Supervision [118.0818807474809]
  This work proposes a principled, probabilistic approach for training explainable multi-hop QA systems without rationale supervision. Our approach performs multi-hop reasoning by explicitly modeling rationales as sets, enabling the model to capture interactions between documents and sentences within a document.
  arXiv  Detail & Related papers  (2023-05-23T16:53:49Z)
• Policy Regularization for Legible Behavior [0.0]
  In Reinforcement Learning interpretability generally means to provide insight into the agent's mechanisms. This paper borrows from the Explainable Planning literature methods that focus on the legibility of the agent. In our formulation, the decision boundary introduced by legibility impacts the states in which the agent's policy returns an action that has high likelihood also in other policies.
  arXiv  Detail & Related papers  (2022-03-08T10:55:46Z)
• Multi-Stage Conversational Passage Retrieval: An Approach to Fusing Term Importance Estimation and Neural Query Rewriting [56.268862325167575]
  We tackle conversational passage retrieval (ConvPR) with query reformulation integrated into a multi-stage ad-hoc IR system. We propose two conversational query reformulation (CQR) methods: (1) term importance estimation and (2) neural query rewriting. For the former, we expand conversational queries using important terms extracted from the conversational context with frequency-based signals. For the latter, we reformulate conversational queries into natural, standalone, human-understandable queries with a pretrained sequence-tosequence model.
  arXiv  Detail & Related papers  (2020-05-05T14:30:20Z)
• Masking Orchestration: Multi-task Pretraining for Multi-role Dialogue Representation Learning [50.5572111079898]
  Multi-role dialogue understanding comprises a wide range of diverse tasks such as question answering, act classification, dialogue summarization etc. While dialogue corpora are abundantly available, labeled data, for specific learning tasks, can be highly scarce and expensive. In this work, we investigate dialogue context representation learning with various types unsupervised pretraining tasks.
  arXiv  Detail & Related papers  (2020-02-27T04:36:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.