Agent Alignment in Evolving Social Norms

• URL: http://arxiv.org/abs/2401.04620v4
• Date: Tue, 20 Feb 2024 03:24:55 GMT
• Title: Agent Alignment in Evolving Social Norms
• Authors: Shimin Li, Tianxiang Sun, Qinyuan Cheng, Xipeng Qiu
• Abstract summary: We propose an evolutionary framework for agent evolution and alignment, named EvolutionaryAgent. In an environment where social norms continuously evolve, agents better adapted to the current social norms will have a higher probability of survival and proliferation. We show that EvolutionaryAgent can align progressively better with the evolving social norms while maintaining its proficiency in general tasks.
• Score: 65.45423591744434
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Agents based on Large Language Models (LLMs) are increasingly permeating various domains of human production and life, highlighting the importance of aligning them with human values. The current alignment of AI systems primarily focuses on passively aligning LLMs through human intervention. However, agents possess characteristics like receiving environmental feedback and self-evolution, rendering the LLM alignment methods inadequate. In response, we propose an evolutionary framework for agent evolution and alignment, named EvolutionaryAgent, which transforms agent alignment into a process of evolution and selection under the principle of survival of the fittest. In an environment where social norms continuously evolve, agents better adapted to the current social norms will have a higher probability of survival and proliferation, while those inadequately aligned dwindle over time. Experimental results assessing the agents from multiple perspectives in aligning with social norms demonstrate that EvolutionaryAgent can align progressively better with the evolving social norms while maintaining its proficiency in general tasks. Effectiveness tests conducted on various open and closed-source LLMs as the foundation for agents also prove the applicability of our approach.

Related papers

• PolicyEvol-Agent: Evolving Policy via Environment Perception and Self-Awareness with Theory of Mind [9.587070290189507]
  PolicyEvol-Agent is a comprehensive framework characterized by systematically acquiring intentions of others. PolicyEvol-Agent integrates a range of cognitive operations with Theory of Mind alongside internal and external perspectives.
  arXiv  Detail & Related papers  (2025-04-20T06:43:23Z)
• Do LLMs trust AI regulation? Emerging behaviour of game-theoretic LLM agents [61.132523071109354]
  This paper investigates the interplay between AI developers, regulators and users, modelling their strategic choices under different regulatory scenarios. Our research identifies emerging behaviours of strategic AI agents, which tend to adopt more "pessimistic" stances than pure game-theoretic agents.
  arXiv  Detail & Related papers  (2025-04-11T15:41:21Z)
• Improving Retrospective Language Agents via Joint Policy Gradient Optimization [57.35348425288859]
  RetroAct is a framework that jointly optimize both task-planning and self-reflective evolution capabilities in language agents. We develop a two-stage joint optimization process that integrates imitation learning and reinforcement learning. We conduct extensive experiments across various testing environments, demonstrating RetroAct has substantial improvements in task performance and decision-making processes.
  arXiv  Detail & Related papers  (2025-03-03T12:54:54Z)
• From Novice to Expert: LLM Agent Policy Optimization via Step-wise Reinforcement Learning [62.54484062185869]
  We introduce StepAgent, which utilizes step-wise reward to optimize the agent's reinforcement learning process. We propose implicit-reward and inverse reinforcement learning techniques to facilitate agent reflection and policy adjustment.
  arXiv  Detail & Related papers  (2024-11-06T10:35:11Z)
• Evolution of Societies via Reinforcement Learning [10.689403855269704]
  We propose a methodology that enables simulating populations of Reinforcement Learning agents at evolutionary scale. We derive a fast, parallelizable implementation of Policy Gradient (PG) and Opponent-Learning Awareness (LOLA) We demonstrate our approach by simulating the evolution of very large populations made of heterogeneous co-learning agents, under both naive and advanced learning strategies.
  arXiv  Detail & Related papers  (2024-10-22T22:49:04Z)
• Moral Alignment for LLM Agents [3.7414804164475983]
  We introduce the design of reward functions that explicitly encode core human values for Reinforcement Learning-based fine-tuning of foundation agent models. We evaluate our approach using the traditional philosophical frameworks of Deontological Ethics and Utilitarianism. We show how moral fine-tuning can be deployed to enable an agent to unlearn a previously developed selfish strategy.
  arXiv  Detail & Related papers  (2024-10-02T15:09:36Z)
• EvoAgent: Towards Automatic Multi-Agent Generation via Evolutionary Algorithms [55.77492625524141]
  EvoAgent is a generic method to automatically extend expert agents to multi-agent systems via the evolutionary algorithm. We show that EvoAgent can automatically generate multiple expert agents and significantly enhance the task-solving capabilities of LLM-based agents.
  arXiv  Detail & Related papers  (2024-06-20T11:49:23Z)
• AgentGym: Evolving Large Language Model-based Agents across Diverse Environments [116.97648507802926]
  Large language models (LLMs) are considered a promising foundation to build such agents. We take the first step towards building generally-capable LLM-based agents with self-evolution ability. We propose AgentGym, a new framework featuring a variety of environments and tasks for broad, real-time, uni-format, and concurrent agent exploration.
  arXiv  Detail & Related papers  (2024-06-06T15:15:41Z)
• A Survey on Self-Evolution of Large Language Models [116.54238664264928]
  Large language models (LLMs) have significantly advanced in various fields and intelligent agent applications. To address this issue, self-evolution approaches that enable LLMs to autonomously acquire, refine, and learn from experiences generated by the model itself are rapidly growing.
  arXiv  Detail & Related papers  (2024-04-22T17:43:23Z)
• The Rise and Potential of Large Language Model Based Agents: A Survey [91.71061158000953]
  Large language models (LLMs) are regarded as potential sparks for Artificial General Intelligence (AGI) We start by tracing the concept of agents from its philosophical origins to its development in AI, and explain why LLMs are suitable foundations for agents. We explore the extensive applications of LLM-based agents in three aspects: single-agent scenarios, multi-agent scenarios, and human-agent cooperation.
  arXiv  Detail & Related papers  (2023-09-14T17:12:03Z)
• When to be critical? Performance and evolvability in different regimes of neural Ising agents [18.536813548129878]
  It has long been hypothesized that operating close to the critical state is beneficial for natural, artificial and their evolutionary systems. We put this hypothesis to test in a system of evolving foraging agents controlled by neural networks. Surprisingly, we find that all populations that discover solutions, evolve to be subcritical.
  arXiv  Detail & Related papers  (2023-03-28T17:57:57Z)
• Co-optimising Robot Morphology and Controller in a Simulated Open-Ended Environment [1.4502611532302039]
  We show how changing the environment, where the agent locomotes, affects the convergence of morphologies. We show that agent-populations evolving in open-endedly evolving environments exhibit larger morphological diversity than agent-populations evolving in hand crafted curricula of environments.
  arXiv  Detail & Related papers  (2021-04-07T11:28:23Z)
• Evolutionary Population Curriculum for Scaling Multi-Agent Reinforcement Learning [37.22210622432453]
  Evolutionary Population Curriculum scales up Multi-Agent Reinforcement Learning (MARL) by progressively increasing the population of training agents in a stage-wise manner. We implement EPC on a popular MARL algorithm, MADDPG, and empirically show that our approach consistently outperforms baselines by a large margin as the number of agents grows exponentially.
  arXiv  Detail & Related papers  (2020-03-23T17:49:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.