Appraisal-Guided Proximal Policy Optimization: Modeling Psychological Disorders in Dynamic Grid World

• URL: http://arxiv.org/abs/2407.20383v1
• Date: Mon, 29 Jul 2024 19:19:54 GMT
• Title: Appraisal-Guided Proximal Policy Optimization: Modeling Psychological Disorders in Dynamic Grid World
• Authors: Hari Prasad, Chinnu Jacob, Imthias Ahamed T. P,
• Abstract summary: We develop a methodology for modeling psychological disorders using Reinforcement Learning (RL) agents. We investigated numerous reward-shaping strategies to simulate psychological disorders and regulate the behavior of the agents. A comparison of various configurations of the modified PPO algorithm identified variants that simulate Anxiety disorder and Obsessive-Compulsive Disorder (OCD)-like behavior in agents.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The integration of artificial intelligence across multiple domains has emphasized the importance of replicating human-like cognitive processes in AI. By incorporating emotional intelligence into AI agents, their emotional stability can be evaluated to enhance their resilience and dependability in critical decision-making tasks. In this work, we develop a methodology for modeling psychological disorders using Reinforcement Learning (RL) agents. We utilized Appraisal theory to train RL agents in a dynamic grid world environment with an Appraisal-Guided Proximal Policy Optimization (AG-PPO) algorithm. Additionally, we investigated numerous reward-shaping strategies to simulate psychological disorders and regulate the behavior of the agents. A comparison of various configurations of the modified PPO algorithm identified variants that simulate Anxiety disorder and Obsessive-Compulsive Disorder (OCD)-like behavior in agents. Furthermore, we compared standard PPO with AG-PPO and its configurations, highlighting the performance improvement in terms of generalization capabilities. Finally, we conducted an analysis of the agents' behavioral patterns in complex test environments to evaluate the associated symptoms corresponding to the psychological disorders. Overall, our work showcases the benefits of the appraisal-guided PPO algorithm over the standard PPO algorithm and the potential to simulate psychological disorders in a controlled artificial environment and evaluate them on RL agents.

Related papers

• PersLLM: A Personified Training Approach for Large Language Models [66.16513246245401]
  We propose PersLLM, integrating psychology-grounded principles of personality: social practice, consistency, and dynamic development. We incorporate personality traits directly into the model parameters, enhancing the model's resistance to induction, promoting consistency, and supporting the dynamic evolution of personality.
  arXiv  Detail & Related papers  (2024-07-17T08:13:22Z)
• Efficient Adaptation in Mixed-Motive Environments via Hierarchical Opponent Modeling and Planning [51.52387511006586]
  We propose Hierarchical Opponent modeling and Planning (HOP), a novel multi-agent decision-making algorithm. HOP is hierarchically composed of two modules: an opponent modeling module that infers others' goals and learns corresponding goal-conditioned policies. HOP exhibits superior few-shot adaptation capabilities when interacting with various unseen agents, and excels in self-play scenarios.
  arXiv  Detail & Related papers  (2024-06-12T08:48:06Z)
• Towards a Psychological Generalist AI: A Survey of Current Applications of Large Language Models and Future Prospects [19.46832545633166]
  The paper emphasizes the importance of performance validation for large-scale AI models. We review the cutting-edge advancements and practical implementations of these expansive models in psychology. These future generalist AI models harbor the potential to substantially curtail labor costs and alleviate social stress.
  arXiv  Detail & Related papers  (2023-12-01T08:35:18Z)
• Adaptive Multi-Agent Deep Reinforcement Learning for Timely Healthcare Interventions [17.405080523382235]
  We propose a novel AI-driven patient monitoring framework using multi-agent deep reinforcement learning (DRL) Our approach deploys multiple learning agents, each dedicated to monitoring a specific physiological feature, such as heart rate, respiration, and temperature. We evaluate the performance of the proposed multi-agent DRL framework using real-world physiological and motion data from two datasets.
  arXiv  Detail & Related papers  (2023-09-20T00:42:08Z)
• Understanding Emergent Behaviours in Multi-Agent Systems with Evolutionary Game Theory [1.0279748604797907]
  This paper summarises some main research directions and challenges tackled in our group, using methods from EGT and ABM. This brief aims to sensitize the reader to EGT based issues, results and prospects, which are accruing in importance for the modeling of minds with machines. In all cases, important open problems in MAS research as viewed or prioritised by the group are described.
  arXiv  Detail & Related papers  (2022-05-15T20:01:48Z)
• POETREE: Interpretable Policy Learning with Adaptive Decision Trees [78.6363825307044]
  POETREE is a novel framework for interpretable policy learning. It builds probabilistic tree policies determining physician actions based on patients' observations and medical history. It outperforms the state-of-the-art on real and synthetic medical datasets.
  arXiv  Detail & Related papers  (2022-03-15T16:50:52Z)
• Model-based Multi-agent Policy Optimization with Adaptive Opponent-wise Rollouts [52.844741540236285]
  This paper investigates the model-based methods in multi-agent reinforcement learning (MARL) We propose a novel decentralized model-based MARL method, named Adaptive Opponent-wise Rollout Policy (AORPO)
  arXiv  Detail & Related papers  (2021-05-07T16:20:22Z)
• AI-Augmented Behavior Analysis for Children with Developmental Disabilities: Building Towards Precision Treatment [2.0324247356209835]
  We present an AI-Augmented Learning and Applied Behavior Analytics (AI-ABA) platform to provide personalized treatment and learning plans. By defining systematic experiments along with automated data collection and analysis, AI-ABA can promote self-regulative behavior.
  arXiv  Detail & Related papers  (2021-02-21T16:15:40Z)
• Unified Models of Human Behavioral Agents in Bandits, Contextual Bandits and RL [28.38826379640553]
  We propose a more general and flexible parametric framework for sequential decision making. Inspired by the known reward processing abnormalities of many mental disorders, our clinically-inspired agents demonstrated interesting behavioral trajectories.
  arXiv  Detail & Related papers  (2020-05-10T01:43:39Z)
• Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observations [88.94162416324505]
  A deep reinforcement learning (DRL) agent observes its states through observations, which may contain natural measurement errors or adversarial noises. Since the observations deviate from the true states, they can mislead the agent into making suboptimal actions. We show that naively applying existing techniques on improving robustness for classification tasks, like adversarial training, is ineffective for many RL tasks.
  arXiv  Detail & Related papers  (2020-03-19T17:59:59Z)
• Decentralized MCTS via Learned Teammate Models [89.24858306636816]
  We present a trainable online decentralized planning algorithm based on decentralized Monte Carlo Tree Search. We show that deep learning and convolutional neural networks can be employed to produce accurate policy approximators.
  arXiv  Detail & Related papers  (2020-03-19T13:10:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.