Rationality based Innate-Values-driven Reinforcement Learning

• URL: http://arxiv.org/abs/2411.09160v1
• Date: Thu, 14 Nov 2024 03:28:02 GMT
• Title: Rationality based Innate-Values-driven Reinforcement Learning
• Authors: Qin Yang,
• Abstract summary: Innate values describe agents' intrinsic motivations, which reflect their inherent interests and preferences to pursue goals. It is an excellent model to describe the innate-values-driven (IV) behaviors of AI agents. This paper proposes a hierarchical reinforcement learning model -- innate-values-driven reinforcement learning model.
• Score: 1.8220718426493654
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Innate values describe agents' intrinsic motivations, which reflect their inherent interests and preferences to pursue goals and drive them to develop diverse skills satisfying their various needs. The essence of reinforcement learning (RL) is learning from interaction based on reward-driven behaviors, much like natural agents. It is an excellent model to describe the innate-values-driven (IV) behaviors of AI agents. Especially developing the awareness of the AI agent through balancing internal and external utilities based on its needs in different tasks is a crucial problem for individuals learning to support AI agents integrating human society with safety and harmony in the long term. This paper proposes a hierarchical compound intrinsic value reinforcement learning model -- innate-values-driven reinforcement learning termed IVRL to describe the complex behaviors of AI agents' interaction. We formulated the IVRL model and proposed two IVRL models: DQN and A2C. By comparing them with benchmark algorithms such as DQN, DDQN, A2C, and PPO in the Role-Playing Game (RPG) reinforcement learning test platform VIZDoom, we demonstrated that rationally organizing various individual needs can effectively achieve better performance.

Related papers

• Dyna-Think: Synergizing Reasoning, Acting, and World Model Simulation in AI Agents [76.86311820866153]
  We propose Dyna-Think, a thinking framework that integrates planning with an internal world model with reasoning and acting to enhance AI agent performance.<n>DIT reconstructs the thinking process of R1 to focus on performing world model simulation relevant to the proposed (and planned) action, and trains the policy using this reconstructed data.<n>DDT uses a two-stage training process to first improve the agent's world modeling ability via objectives such as state prediction or critique generation, and then improve the agent's action via policy training.
  arXiv  Detail & Related papers  (2025-05-31T00:10:18Z)
• DSADF: Thinking Fast and Slow for Decision Making [9.84593001541736]
  We propose a Dual-System Adaptive Decision Framework (DSADF) to integrate two complementary modules: System 1, comprising an RL agent and a memory space for fast and intuitive decision making, and System 2, driven by a VLM for deep and analytical reasoning.
  arXiv  Detail & Related papers  (2025-05-13T02:58:04Z)
• Interactive Double Deep Q-network: Integrating Human Interventions and Evaluative Predictions in Reinforcement Learning of Autonomous Driving [16.379623042562415]
  This study introduces Interactive Double Deep Q-network (iDDQN), a Human-in-the-Loop (HITL) approach that enhances Reinforcement Learning (RL)<n>Our proposed iDDQN method modifies the Q-value update equation to integrate human and agent actions, establishing a collaborative approach for policy development.<n> Empirical results in simulated autonomous driving scenarios demonstrate that iDDQN outperforms established approaches.
  arXiv  Detail & Related papers  (2025-04-28T05:25:18Z)
• A Nature-Inspired Colony of Artificial Intelligence System with Fast, Detailed, and Organized Learner Agents for Enhancing Diversity and Quality [0.0]
  We present an approach that builds a CNN-based colony of AI agents to serve as a single system. The proposed system impersonates the natural environment of a biological system, like an ant colony or a human colony. The evolution of fast, detailed, and organized learners in the colony of AI is achieved by introducing a unique one-to-one mapping.
  arXiv  Detail & Related papers  (2025-04-07T12:13:14Z)
• Agentic Knowledgeable Self-awareness [79.25908923383776]
  KnowSelf is a data-centric approach that applies agents with knowledgeable self-awareness like humans.<n>Our experiments demonstrate that KnowSelf can outperform various strong baselines on different tasks and models with minimal use of external knowledge.
  arXiv  Detail & Related papers  (2025-04-04T16:03:38Z)
• Autotelic Reinforcement Learning: Exploring Intrinsic Motivations for Skill Acquisition in Open-Ended Environments [1.104960878651584]
  This paper presents a comprehensive overview of autotelic Reinforcement Learning (RL), emphasizing the role of intrinsic motivations in the open-ended formation of skill repertoires.<n>We delineate the distinctions between knowledge-based and competence-based intrinsic motivations, illustrating how these concepts inform the development of autonomous agents capable of generating and pursuing self-defined goals.
  arXiv  Detail & Related papers  (2025-02-06T14:37:46Z)
• Simulating Human-like Daily Activities with Desire-driven Autonomy [25.380194192389492]
  We introduce a Desire-driven Autonomous Agent (D2A) that can enable a large language model (LLM) to autonomously propose and select tasks. At each step, the agent evaluates the value of its current state, proposes a set of candidate activities, and selects the one that best aligns with its intrinsic motivations.
  arXiv  Detail & Related papers  (2024-12-09T12:21:20Z)
• A Unifying Framework for Action-Conditional Self-Predictive Reinforcement Learning [48.59516337905877]
  Learning a good representation is a crucial challenge for Reinforcement Learning (RL) agents. Recent work has developed theoretical insights into these algorithms. We take a step towards bridging the gap between theory and practice by analyzing an action-conditional self-predictive objective.
  arXiv  Detail & Related papers  (2024-06-04T07:22:12Z)
• Position Paper: Agent AI Towards a Holistic Intelligence [53.35971598180146]
  We emphasize developing Agent AI -- an embodied system that integrates large foundation models into agent actions. In this paper, we propose a novel large action model to achieve embodied intelligent behavior, the Agent Foundation Model.
  arXiv  Detail & Related papers  (2024-02-28T16:09:56Z)
• Innate-Values-driven Reinforcement Learning for Cooperative Multi-Agent Systems [1.8220718426493654]
  Innate values describe agents' intrinsic motivations, which reflect their inherent interests and preferences to pursue goals. The essence of reinforcement learning (RL) is learning from interaction based on reward-driven (such as utilities) behaviors. This paper proposes a hierarchical compound value reinforcement learning model -- intrinsic-driven reinforcement learning -- to describe the complex behaviors of multi-agent interaction.
  arXiv  Detail & Related papers  (2024-01-10T22:51:10Z)
• Pangu-Agent: A Fine-Tunable Generalist Agent with Structured Reasoning [50.47568731994238]
  Key method for creating Artificial Intelligence (AI) agents is Reinforcement Learning (RL) This paper presents a general framework model for integrating and learning structured reasoning into AI agents' policies.
  arXiv  Detail & Related papers  (2023-12-22T17:57:57Z)
• DCIR: Dynamic Consistency Intrinsic Reward for Multi-Agent Reinforcement Learning [84.22561239481901]
  We propose a new approach that enables agents to learn whether their behaviors should be consistent with that of other agents. We evaluate DCIR in multiple environments including Multi-agent Particle, Google Research Football and StarCraft II Micromanagement.
  arXiv  Detail & Related papers  (2023-12-10T06:03:57Z)
• Interactive Autonomous Navigation with Internal State Inference and Interactivity Estimation [58.21683603243387]
  We propose three auxiliary tasks with relational-temporal reasoning and integrate them into the standard Deep Learning framework. These auxiliary tasks provide additional supervision signals to infer the behavior patterns other interactive agents. Our approach achieves robust and state-of-the-art performance in terms of standard evaluation metrics.
  arXiv  Detail & Related papers  (2023-11-27T18:57:42Z)
• Learning in Cooperative Multiagent Systems Using Cognitive and Machine Models [1.0742675209112622]
  Multi-Agent Systems (MAS) are critical for many applications requiring collaboration and coordination with humans. One major challenge is the simultaneous learning and interaction of independent agents in dynamic environments. We propose three variants of Multi-Agent IBL models (MAIBL) We demonstrate that the MAIBL models exhibit faster learning and achieve better coordination in a dynamic CMOTP task with various settings of rewards compared to current MADRL models.
  arXiv  Detail & Related papers  (2023-08-18T00:39:06Z)
• Learning to Walk Autonomously via Reset-Free Quality-Diversity [73.08073762433376]
  Quality-Diversity algorithms can discover large and complex behavioural repertoires consisting of both diverse and high-performing skills. Existing QD algorithms need large numbers of evaluations as well as episodic resets, which require manual human supervision and interventions. This paper proposes Reset-Free Quality-Diversity optimization (RF-QD) as a step towards autonomous learning for robotics in open-ended environments.
  arXiv  Detail & Related papers  (2022-04-07T14:07:51Z)
• Modeling Bounded Rationality in Multi-Agent Simulations Using Rationally Inattentive Reinforcement Learning [85.86440477005523]
  We study more human-like RL agents which incorporate an established model of human-irrationality, the Rational Inattention (RI) model. RIRL models the cost of cognitive information processing using mutual information. We show that using RIRL yields a rich spectrum of new equilibrium behaviors that differ from those found under rational assumptions.
  arXiv  Detail & Related papers  (2022-01-18T20:54:00Z)
• An active inference model of collective intelligence [0.0]
  This paper posits a minimal agent-based model that simulates the relationship between local individual-level interaction and collective intelligence. Results show that stepwise cognitive transitions increase system performance by providing complementary mechanisms for alignment between agents' local and global optima.
  arXiv  Detail & Related papers  (2021-04-02T14:32:01Z)
• Reinforcement Learning through Active Inference [62.997667081978825]
  We show how ideas from active inference can augment traditional reinforcement learning approaches. We develop and implement a novel objective for decision making, which we term the free energy of the expected future. We demonstrate that the resulting algorithm successfully exploration and exploitation, simultaneously achieving robust performance on several challenging RL benchmarks with sparse, well-shaped, and no rewards.
  arXiv  Detail & Related papers  (2020-02-28T10:28:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.