Mitigating Negative Side Effects via Environment Shaping

• URL: http://arxiv.org/abs/2102.07017v1
• Date: Sat, 13 Feb 2021 22:15:00 GMT
• Title: Mitigating Negative Side Effects via Environment Shaping
• Authors: Sandhya Saisubramanian and Shlomo Zilberstein
• Abstract summary: Agents operating in unstructured environments often produce negative side effects (NSE) We present an algorithm to solve this problem and analyze its theoretical properties. Empirical evaluation of our approach shows that the proposed framework can successfully mitigate NSE, without affecting the agent's ability to complete its assigned task.
• Score: 27.400267388362654
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Agents operating in unstructured environments often produce negative side effects (NSE), which are difficult to identify at design time. While the agent can learn to mitigate the side effects from human feedback, such feedback is often expensive and the rate of learning is sensitive to the agent's state representation. We examine how humans can assist an agent, beyond providing feedback, and exploit their broader scope of knowledge to mitigate the impacts of NSE. We formulate this problem as a human-agent team with decoupled objectives. The agent optimizes its assigned task, during which its actions may produce NSE. The human shapes the environment through minor reconfiguration actions so as to mitigate the impacts of the agent's side effects, without affecting the agent's ability to complete its assigned task. We present an algorithm to solve this problem and analyze its theoretical properties. Through experiments with human subjects, we assess the willingness of users to perform minor environment modifications to mitigate the impacts of NSE. Empirical evaluation of our approach shows that the proposed framework can successfully mitigate NSE, without affecting the agent's ability to complete its assigned task.

Related papers

• Counterfactual Effect Decomposition in Multi-Agent Sequential Decision Making [9.469649321687928]
  We introduce a novel causal explanation formula that decomposes the counterfactual effect by attributing to each agent and state variable a score reflecting their respective contributions to the effect. We show that the total counterfactual effect of an agent's action can be decomposed into two components: one measuring the effect that propagates through all subsequent agents' actions and another related to the effect that propagates through the state transitions.
  arXiv  Detail & Related papers  (2024-10-16T13:20:35Z)
• Caution for the Environment: Multimodal Agents are Susceptible to Environmental Distractions [68.92637077909693]
  This paper investigates the faithfulness of multimodal large language model (MLLM) agents in the graphical user interface (GUI) environment. A general setting is proposed where both the user and the agent are benign, and the environment, while not malicious, contains unrelated content. Experimental results reveal that even the most powerful models, whether generalist agents or specialist GUI agents, are susceptible to distractions.
  arXiv  Detail & Related papers  (2024-08-05T15:16:22Z)
• Improving Zero-Shot ObjectNav with Generative Communication [60.84730028539513]
  We propose a new method for improving zero-shot ObjectNav. Our approach takes into account that the ground agent may have limited and sometimes obstructed view.
  arXiv  Detail & Related papers  (2024-08-03T22:55:26Z)
• External Model Motivated Agents: Reinforcement Learning for Enhanced Environment Sampling [3.536024441537599]
  Unlike reinforcement learning (RL) agents, humans remain capable multitaskers in changing environments. We propose an agent influence framework for RL agents to improve the adaptation efficiency of external models in changing environments. Our results show that our method outperforms the baselines in terms of external model adaptation on metrics that measure both efficiency and performance.
  arXiv  Detail & Related papers  (2024-06-28T23:31:22Z)
• Learning Actionable Counterfactual Explanations in Large State Spaces [16.30292272064278]
  We consider settings where optimal CFEs correspond to solutions of weighted set cover problems. In this work, we provide a deep-network learning procedure that we show experimentally is able to achieve strong performance. Our problem can also be viewed as one of learning an optimal policy in a family of large but deterministic Markov Decision Processes.
  arXiv  Detail & Related papers  (2024-04-25T20:49:03Z)
• AntEval: Evaluation of Social Interaction Competencies in LLM-Driven Agents [65.16893197330589]
  Large Language Models (LLMs) have demonstrated their ability to replicate human behaviors across a wide range of scenarios. However, their capability in handling complex, multi-character social interactions has yet to be fully explored. We introduce the Multi-Agent Interaction Evaluation Framework (AntEval), encompassing a novel interaction framework and evaluation methods.
  arXiv  Detail & Related papers  (2024-01-12T11:18:00Z)
• Feedback Effect in User Interaction with Intelligent Assistants: Delayed Engagement, Adaption and Drop-out [9.205174767678365]
  This paper identifies and quantifies the feedback effect, a novel component in IA-user interactions. We show that unhelpful responses from the IA cause users to delay or reduce subsequent interactions. As users discover the limitations of the IA's understanding and functional capabilities, they learn to adjust the scope and wording of their requests.
  arXiv  Detail & Related papers  (2023-03-17T21:39:33Z)
• Formalizing the Problem of Side Effect Regularization [81.97441214404247]
  We propose a formal criterion for side effect regularization via the assistance game framework. In these games, the agent solves a partially observable Markov decision process. We show that this POMDP is solved by trading off the proxy reward with the agent's ability to achieve a range of future tasks.
  arXiv  Detail & Related papers  (2022-06-23T16:36:13Z)
• Inverse Online Learning: Understanding Non-Stationary and Reactionary Policies [79.60322329952453]
  We show how to develop interpretable representations of how agents make decisions. By understanding the decision-making processes underlying a set of observed trajectories, we cast the policy inference problem as the inverse to this online learning problem. We introduce a practical algorithm for retrospectively estimating such perceived effects, alongside the process through which agents update them. Through application to the analysis of UNOS organ donation acceptance decisions, we demonstrate that our approach can bring valuable insights into the factors that govern decision processes and how they change over time.
  arXiv  Detail & Related papers  (2022-03-14T17:40:42Z)
• Information is Power: Intrinsic Control via Information Capture [110.3143711650806]
  We argue that a compact and general learning objective is to minimize the entropy of the agent's state visitation estimated using a latent state-space model. This objective induces an agent to both gather information about its environment, corresponding to reducing uncertainty, and to gain control over its environment, corresponding to reducing the unpredictability of future world states.
  arXiv  Detail & Related papers  (2021-12-07T18:50:42Z)
• On the Sensory Commutativity of Action Sequences for Embodied Agents [2.320417845168326]
  We study perception for embodied agents under the mathematical formalism of group theory. We introduce the Sensory Commutativity Probability criterion which measures how much an agent's degree of freedom affects the environment. We empirically illustrate how SCP and the commutative properties of action sequences can be used to learn about objects in the environment.
  arXiv  Detail & Related papers  (2020-02-13T16:58:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.