Play with Emotion: Affect-Driven Reinforcement Learning

• URL: http://arxiv.org/abs/2208.12622v1
• Date: Fri, 26 Aug 2022 12:28:24 GMT
• Title: Play with Emotion: Affect-Driven Reinforcement Learning
• Authors: Matthew Barthet, Ahmed Khalifa, Antonios Liapis and Georgios N. Yannakakis
• Abstract summary: This paper introduces a paradigm shift by viewing the task of affect modeling as a reinforcement learning process. We test our hypotheses in a racing game by training Go-Blend agents to model human demonstrations of arousal and behavior.
• Score: 3.611888922173257
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper introduces a paradigm shift by viewing the task of affect modeling as a reinforcement learning (RL) process. According to the proposed paradigm, RL agents learn a policy (i.e. affective interaction) by attempting to maximize a set of rewards (i.e. behavioral and affective patterns) via their experience with their environment (i.e. context). Our hypothesis is that RL is an effective paradigm for interweaving affect elicitation and manifestation with behavioral and affective demonstrations. Importantly, our second hypothesis-building on Damasio's somatic marker hypothesis-is that emotion can be the facilitator of decision-making. We test our hypotheses in a racing game by training Go-Blend agents to model human demonstrations of arousal and behavior; Go-Blend is a modified version of the Go-Explore algorithm which has recently showcased supreme performance in hard exploration tasks. We first vary the arousal-based reward function and observe agents that can effectively display a palette of affect and behavioral patterns according to the specified reward. Then we use arousal-based state selection mechanisms in order to bias the strategies that Go-Blend explores. Our findings suggest that Go-Blend not only is an efficient affect modeling paradigm but, more importantly, affect-driven RL improves exploration and yields higher performing agents, validating Damasio's hypothesis in the domain of games.

Related papers

• Variable-Agnostic Causal Exploration for Reinforcement Learning [56.52768265734155]
  We introduce a novel framework, Variable-Agnostic Causal Exploration for Reinforcement Learning (VACERL) Our approach automatically identifies crucial observation-action steps associated with key variables using attention mechanisms. It constructs the causal graph connecting these steps, which guides the agent towards observation-action pairs with greater causal influence on task completion.
  arXiv  Detail & Related papers  (2024-07-17T09:45:27Z)
• Action Inference by Maximising Evidence: Zero-Shot Imitation from Observation with World Models [9.583751440005118]
  We propose Action Inference by Maximising Evidence (AIME) to replicate this behaviour using world models. AIME consists of two distinct phases. In the first phase, the agent learns a world model from its past experience to understand its own body by maximising the ELBO. While in the second phase, the agent is given some observation-only demonstrations of an expert performing a novel task and tries to imitate the expert's behaviour. Our method is "zero-shot" in the sense that it does not require further training for the world model or online interactions with the environment after given the demonstration
  arXiv  Detail & Related papers  (2023-12-04T16:43:36Z)
• Interpretable Imitation Learning with Dynamic Causal Relations [65.18456572421702]
  We propose to expose captured knowledge in the form of a directed acyclic causal graph. We also design this causal discovery process to be state-dependent, enabling it to model the dynamics in latent causal graphs. The proposed framework is composed of three parts: a dynamic causal discovery module, a causality encoding module, and a prediction module, and is trained in an end-to-end manner.
  arXiv  Detail & Related papers  (2023-09-30T20:59:42Z)
• Learning Goal-based Movement via Motivational-based Models in Cognitive Mobile Robots [58.720142291102135]
  Humans have needs motivating their behavior according to intensity and context. We also create preferences associated with each action's perceived pleasure, which is susceptible to changes over time. This makes decision-making more complex, requiring learning to balance needs and preferences according to the context.
  arXiv  Detail & Related papers  (2023-02-20T04:52:24Z)
• Basis for Intentions: Efficient Inverse Reinforcement Learning using Past Experience [89.30876995059168]
  inverse reinforcement learning (IRL) -- inferring the reward function of an agent from observing its behavior. This paper addresses the problem of IRL -- inferring the reward function of an agent from observing its behavior.
  arXiv  Detail & Related papers  (2022-08-09T17:29:49Z)
• Modelling Behaviour Change using Cognitive Agent Simulations [0.0]
  This paper presents work-in-progress research to apply selected behaviour change theories to simulated agents. The research is focusing on complex agent architectures required for self-determined goal achievement in adverse circumstances.
  arXiv  Detail & Related papers  (2021-10-16T19:19:08Z)
• Go-Blend behavior and affect [2.323282558557423]
  This paper proposes a paradigm shift for affective computing by viewing the affect modeling task as a reinforcement learning process. In this initial study, we test our framework in an arcade game by training Go-Explore agents to both play optimally and attempt to mimic human demonstrations of arousal.
  arXiv  Detail & Related papers  (2021-09-24T17:04:30Z)
• Antagonistic Crowd Simulation Model Integrating Emotion Contagion and Deep Reinforcement Learning [19.60008056384961]
  The mechanism of complex emotion influencing decision making, especially in the environment of sudden confrontation, has not yet been explored clearly. We propose one new antagonistic crowd simulation model by combing emotional contagion and deep reinforcement learning. The results prove that emotions have a vital impact on the group combat, and positive emotional states are more conducive to combat.
  arXiv  Detail & Related papers  (2021-04-29T01:18:13Z)
• Disturbing Reinforcement Learning Agents with Corrupted Rewards [62.997667081978825]
  We analyze the effects of different attack strategies based on reward perturbations on reinforcement learning algorithms. We show that smoothly crafting adversarial rewards are able to mislead the learner, and that using low exploration probability values, the policy learned is more robust to corrupt rewards.
  arXiv  Detail & Related papers  (2021-02-12T15:53:48Z)
• Learning intuitive physics and one-shot imitation using state-action-prediction self-organizing maps [0.0]
  Humans learn by exploration and imitation, build causal models of the world, and use both to flexibly solve new tasks. We suggest a simple but effective unsupervised model which develops such characteristics. We demonstrate its performance on a set of several related, but different one-shot imitation tasks, which the agent flexibly solves in an active inference style.
  arXiv  Detail & Related papers  (2020-07-03T12:29:11Z)
• Intrinsic Motivation for Encouraging Synergistic Behavior [55.10275467562764]
  We study the role of intrinsic motivation as an exploration bias for reinforcement learning in sparse-reward synergistic tasks. Our key idea is that a good guiding principle for intrinsic motivation in synergistic tasks is to take actions which affect the world in ways that would not be achieved if the agents were acting on their own.
  arXiv  Detail & Related papers  (2020-02-12T19:34:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.