Interrogating the Black Box: Transparency through Information-Seeking Dialogues

• URL: http://arxiv.org/abs/2102.04714v1
• Date: Tue, 9 Feb 2021 09:14:04 GMT
• Title: Interrogating the Black Box: Transparency through Information-Seeking Dialogues
• Authors: Andrea Aler Tubella, Andreas Theodorou and Juan Carlos Nieves
• Abstract summary: We propose to construct an investigator agent to query a learning agent to investigate its adherence to an ethical policy. This formal dialogue framework is the main contribution of this paper. We argue that the introduced formal dialogue framework opens many avenues both in the area of compliance checking and in the analysis of properties of opaque systems.
• Score: 9.281671380673306
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper is preoccupied with the following question: given a (possibly opaque) learning system, how can we understand whether its behaviour adheres to governance constraints? The answer can be quite simple: we just need to "ask" the system about it. We propose to construct an investigator agent to query a learning agent -- the suspect agent -- to investigate its adherence to a given ethical policy in the context of an information-seeking dialogue, modeled in formal argumentation settings. This formal dialogue framework is the main contribution of this paper. Through it, we break down compliance checking mechanisms into three modular components, each of which can be tailored to various needs in a vast amount of ways: an investigator agent, a suspect agent, and an acceptance protocol determining whether the responses of the suspect agent comply with the policy. This acceptance protocol presents a fundamentally different approach to aggregation: rather than using quantitative methods to deal with the non-determinism of a learning system, we leverage the use of argumentation semantics to investigate the notion of properties holding consistently. Overall, we argue that the introduced formal dialogue framework opens many avenues both in the area of compliance checking and in the analysis of properties of opaque systems.

Related papers

• Understanding Understanding: A Pragmatic Framework Motivated by Large Language Models [13.279760256875127]
  In Turing-test fashion, the framework is based solely on the agent's performance, and specifically on how well it answers questions. We show how high confidence can be achieved via random sampling and the application of probabilistic confidence bounds.
  arXiv  Detail & Related papers  (2024-06-16T13:37:08Z)
• Leveraging Counterfactual Paths for Contrastive Explanations of POMDP Policies [2.4332936182093197]
  XAI aims to reduce confusion and foster trust in systems by providing explanations of agent behavior. POMDPs provide a flexible framework capable of reasoning over transition and state uncertainty. This work investigates the use of user-provided counterfactuals to generate contrastive explanations of POMDP policies.
  arXiv  Detail & Related papers  (2024-03-28T18:19:38Z)
• Clarify When Necessary: Resolving Ambiguity Through Interaction with LMs [58.620269228776294]
  We propose a task-agnostic framework for resolving ambiguity by asking users clarifying questions. We evaluate systems across three NLP applications: question answering, machine translation and natural language inference. We find that intent-sim is robust, demonstrating improvements across a wide range of NLP tasks and LMs.
  arXiv  Detail & Related papers  (2023-11-16T00:18:50Z)
• A Unifying Framework for Learning Argumentation Semantics [50.69905074548764]
  We present a novel framework, which uses an Inductive Logic Programming approach to learn the acceptability semantics for several abstract and structured argumentation frameworks in an interpretable way. Our framework outperforms existing argumentation solvers, thus opening up new future research directions in the area of formal argumentation and human-machine dialogues.
  arXiv  Detail & Related papers  (2023-10-18T20:18:05Z)
• HOP, UNION, GENERATE: Explainable Multi-hop Reasoning without Rationale Supervision [118.0818807474809]
  This work proposes a principled, probabilistic approach for training explainable multi-hop QA systems without rationale supervision. Our approach performs multi-hop reasoning by explicitly modeling rationales as sets, enabling the model to capture interactions between documents and sentences within a document.
  arXiv  Detail & Related papers  (2023-05-23T16:53:49Z)
• Establishing Shared Query Understanding in an Open Multi-Agent System [1.2031796234206138]
  We propose a method that allows to develop shared understanding between two agents for the purpose of performing a task that requires cooperation. Our method focuses on efficiently establishing successful task-oriented communication in an open multi-agent system.
  arXiv  Detail & Related papers  (2023-05-16T11:07:05Z)
• Policy Regularization for Legible Behavior [0.0]
  In Reinforcement Learning interpretability generally means to provide insight into the agent's mechanisms. This paper borrows from the Explainable Planning literature methods that focus on the legibility of the agent. In our formulation, the decision boundary introduced by legibility impacts the states in which the agent's policy returns an action that has high likelihood also in other policies.
  arXiv  Detail & Related papers  (2022-03-08T10:55:46Z)
• Towards Identifying Social Bias in Dialog Systems: Frame, Datasets, and Benchmarks [95.29345070102045]
  In this paper, we focus our investigation on social bias detection of dialog safety problems. We first propose a novel Dial-Bias Frame for analyzing the social bias in conversations pragmatically. We introduce CDail-Bias dataset that is the first well-annotated Chinese social bias dialog dataset.
  arXiv  Detail & Related papers  (2022-02-16T11:59:29Z)
• Self-supervised Dialogue Learning for Spoken Conversational Question Answering [29.545937716796082]
  In spoken conversational question answering (SCQA), the answer to the corresponding question is generated by retrieving and then analyzing a fixed spoken document, including multi-part conversations. We introduce a self-supervised learning approach, including incoherence discrimination, insertion detection, and question prediction, to explicitly capture the coreference resolution and dialogue coherence. Our proposed method provides more coherent, meaningful, and appropriate responses, yielding superior performance gains compared to the original pre-trained language models.
  arXiv  Detail & Related papers  (2021-06-04T00:09:38Z)
• Multi-Stage Conversational Passage Retrieval: An Approach to Fusing Term Importance Estimation and Neural Query Rewriting [56.268862325167575]
  We tackle conversational passage retrieval (ConvPR) with query reformulation integrated into a multi-stage ad-hoc IR system. We propose two conversational query reformulation (CQR) methods: (1) term importance estimation and (2) neural query rewriting. For the former, we expand conversational queries using important terms extracted from the conversational context with frequency-based signals. For the latter, we reformulate conversational queries into natural, standalone, human-understandable queries with a pretrained sequence-tosequence model.
  arXiv  Detail & Related papers  (2020-05-05T14:30:20Z)
• Masking Orchestration: Multi-task Pretraining for Multi-role Dialogue Representation Learning [50.5572111079898]
  Multi-role dialogue understanding comprises a wide range of diverse tasks such as question answering, act classification, dialogue summarization etc. While dialogue corpora are abundantly available, labeled data, for specific learning tasks, can be highly scarce and expensive. In this work, we investigate dialogue context representation learning with various types unsupervised pretraining tasks.
  arXiv  Detail & Related papers  (2020-02-27T04:36:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.