Methods and Mechanisms for Interactive Novelty Handling in Adversarial Environments

• URL: http://arxiv.org/abs/2302.14208v1
• Date: Tue, 28 Feb 2023 00:05:48 GMT
• Title: Methods and Mechanisms for Interactive Novelty Handling in Adversarial Environments
• Authors: Tung Thai, Ming Shen, Mayank Garg, Ayush Kalani, Nakul Vaidya, Utkarsh Soni, Mudit Verma, Sriram Gopalakrishnan, Chitta Baral, Subbarao Kambhampati, Jivko Sinapov, and Matthias Scheutz
• Abstract summary: We introduce general methods and architectural mechanisms for detecting and characterizing different types of novelties. We demonstrate the effectiveness of the proposed methods in evaluations performed by a third party in the adversarial multi-agent board game Monopoly.
• Score: 32.175953686781284
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Learning to detect, characterize and accommodate novelties is a challenge that agents operating in open-world domains need to address to be able to guarantee satisfactory task performance. Certain novelties (e.g., changes in environment dynamics) can interfere with the performance or prevent agents from accomplishing task goals altogether. In this paper, we introduce general methods and architectural mechanisms for detecting and characterizing different types of novelties, and for building an appropriate adaptive model to accommodate them utilizing logical representations and reasoning methods. We demonstrate the effectiveness of the proposed methods in evaluations performed by a third party in the adversarial multi-agent board game Monopoly. The results show high novelty detection and accommodation rates across a variety of novelty types, including changes to the rules of the game, as well as changes to the agent's action capabilities.

Related papers

• Active Legibility in Multiagent Reinforcement Learning [3.7828554251478734]
  The legibility-oriented framework allows agents to conduct legible actions so as to help others optimise their behaviors. The experimental results demonstrate that the new framework is more efficient and costs less training time compared to several multiagent reinforcement learning algorithms.
  arXiv  Detail & Related papers  (2024-10-28T12:15:49Z)
• Efficient Adaptation in Mixed-Motive Environments via Hierarchical Opponent Modeling and Planning [51.52387511006586]
  We propose Hierarchical Opponent modeling and Planning (HOP), a novel multi-agent decision-making algorithm. HOP is hierarchically composed of two modules: an opponent modeling module that infers others' goals and learns corresponding goal-conditioned policies. HOP exhibits superior few-shot adaptation capabilities when interacting with various unseen agents, and excels in self-play scenarios.
  arXiv  Detail & Related papers  (2024-06-12T08:48:06Z)
• Learning to Operate in Open Worlds by Adapting Planning Models [12.513121330508477]
  Planning agents are ill-equipped to act in novel situations in which their domain model no longer accurately represents the world. We introduce an approach for such agents operating in open worlds that detects the presence of novelties and effectively adapts their domain models.
  arXiv  Detail & Related papers  (2023-03-24T21:04:16Z)
• Intrinsic Motivation in Model-based Reinforcement Learning: A Brief Review [77.34726150561087]
  This review considers the existing methods for determining intrinsic motivation based on the world model obtained by the agent. The proposed unified framework describes the architecture of agents using a world model and intrinsic motivation to improve learning.
  arXiv  Detail & Related papers  (2023-01-24T15:13:02Z)
• Inferring Versatile Behavior from Demonstrations by Matching Geometric Descriptors [72.62423312645953]
  Humans intuitively solve tasks in versatile ways, varying their behavior in terms of trajectory-based planning and for individual steps. Current Imitation Learning algorithms often only consider unimodal expert demonstrations and act in a state-action-based setting. Instead, we combine a mixture of movement primitives with a distribution matching objective to learn versatile behaviors that match the expert's behavior and versatility.
  arXiv  Detail & Related papers  (2022-10-17T16:42:59Z)
• REPTILE: A Proactive Real-Time Deep Reinforcement Learning Self-adaptive Framework [0.6335848702857039]
  A general framework is proposed to support the development of software systems that are able to adapt their behaviour according to the operating environment changes. The proposed approach, named REPTILE, works in a complete proactive manner and relies on Deep Reinforcement Learning-based agents to react to events. In our framework, two types of novelties are taken into account: those related to the context/environment and those related to the physical architecture itself. The framework, predicting those novelties before their occurrence, extracts time-changing models of the environment and uses a suitable Markov Decision Process to deal with the real-time setting.
  arXiv  Detail & Related papers  (2022-03-28T12:38:08Z)
• Audio-Adaptive Activity Recognition Across Video Domains [112.46638682143065]
  We leverage activity sounds for domain adaptation as they have less variance across domains and can reliably indicate which activities are not happening. We propose an audio-adaptive encoder and associated learning methods that discriminatively adjust the visual feature representation. We also introduce the new task of actor shift, with a corresponding audio-visual dataset, to challenge our method with situations where the activity appearance changes dramatically.
  arXiv  Detail & Related papers  (2022-03-27T08:15:20Z)
• NovGrid: A Flexible Grid World for Evaluating Agent Response to Novelty [8.705624336757461]
  We introduce NovGrid, a novelty generation framework built on MiniGrid. Along with the core NovGrid we provide exemplar novelties aligned with our ontology and instantiate them as novelty templates. We present a set of metrics built into our framework for the evaluation of novelty-adaptation-enabled machine-learning techniques.
  arXiv  Detail & Related papers  (2022-03-23T01:06:04Z)
• Simulating and classifying behavior in adversarial environments based on action-state traces: an application to money laundering [18.625578105241]
  We present a novel way of approaching these types of applications, in particular in the context of Anti-Money Laundering. We provide a mechanism through which diverse, realistic and new unobserved behavior may be generated to discover potential unobserved adversarial actions.
  arXiv  Detail & Related papers  (2020-11-03T16:30:53Z)
• Behavior Priors for Efficient Reinforcement Learning [97.81587970962232]
  We consider how information and architectural constraints can be combined with ideas from the probabilistic modeling literature to learn behavior priors. We discuss how such latent variable formulations connect to related work on hierarchical reinforcement learning (HRL) and mutual information and curiosity based objectives. We demonstrate the effectiveness of our framework by applying it to a range of simulated continuous control domains.
  arXiv  Detail & Related papers  (2020-10-27T13:17:18Z)
• End-to-End Learning and Intervention in Games [60.41921763076017]
  We provide a unified framework for learning and intervention in games. We propose two approaches, respectively based on explicit and implicit differentiation. The analytical results are validated using several real-world problems.
  arXiv  Detail & Related papers  (2020-10-26T18:39:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.