Agent Alignment in Evolving Social Norms

• URL: http://arxiv.org/abs/2401.04620v4
• Date: Tue, 20 Feb 2024 03:24:55 GMT
• Title: Agent Alignment in Evolving Social Norms
• Authors: Shimin Li, Tianxiang Sun, Qinyuan Cheng, Xipeng Qiu
• Abstract summary: We propose an evolutionary framework for agent evolution and alignment, named EvolutionaryAgent. In an environment where social norms continuously evolve, agents better adapted to the current social norms will have a higher probability of survival and proliferation. We show that EvolutionaryAgent can align progressively better with the evolving social norms while maintaining its proficiency in general tasks.
• Score: 65.45423591744434
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Agents based on Large Language Models (LLMs) are increasingly permeating various domains of human production and life, highlighting the importance of aligning them with human values. The current alignment of AI systems primarily focuses on passively aligning LLMs through human intervention. However, agents possess characteristics like receiving environmental feedback and self-evolution, rendering the LLM alignment methods inadequate. In response, we propose an evolutionary framework for agent evolution and alignment, named EvolutionaryAgent, which transforms agent alignment into a process of evolution and selection under the principle of survival of the fittest. In an environment where social norms continuously evolve, agents better adapted to the current social norms will have a higher probability of survival and proliferation, while those inadequately aligned dwindle over time. Experimental results assessing the agents from multiple perspectives in aligning with social norms demonstrate that EvolutionaryAgent can align progressively better with the evolving social norms while maintaining its proficiency in general tasks. Effectiveness tests conducted on various open and closed-source LLMs as the foundation for agents also prove the applicability of our approach.

Related papers

• Cultural Evolution of Cooperation among LLM Agents [1.4261864659766343]
  We study whether a "society" of AI agents can learn mutually beneficial social norms in the face of incentives to defect. We find that the evolution of cooperation differs markedly across base models.
  arXiv  Detail & Related papers  (2024-12-13T16:45:49Z)
• From Novice to Expert: LLM Agent Policy Optimization via Step-wise Reinforcement Learning [62.54484062185869]
  We introduce StepAgent, which utilizes step-wise reward to optimize the agent's reinforcement learning process. We propose implicit-reward and inverse reinforcement learning techniques to facilitate agent reflection and policy adjustment.
  arXiv  Detail & Related papers  (2024-11-06T10:35:11Z)
• Moral Alignment for LLM Agents [3.7414804164475983]
  We introduce the design of reward functions that explicitly encode core human values for Reinforcement Learning-based fine-tuning of foundation agent models. We evaluate our approach using the traditional philosophical frameworks of Deontological Ethics and Utilitarianism. We show how moral fine-tuning can be deployed to enable an agent to unlearn a previously developed selfish strategy.
  arXiv  Detail & Related papers  (2024-10-02T15:09:36Z)
• EvoAgent: Towards Automatic Multi-Agent Generation via Evolutionary Algorithms [55.77492625524141]
  EvoAgent is a generic method to automatically extend expert agents to multi-agent systems via the evolutionary algorithm. We show that EvoAgent can automatically generate multiple expert agents and significantly enhance the task-solving capabilities of LLM-based agents.
  arXiv  Detail & Related papers  (2024-06-20T11:49:23Z)
• AgentGym: Evolving Large Language Model-based Agents across Diverse Environments [116.97648507802926]
  Large language models (LLMs) are considered a promising foundation to build such agents. We take the first step towards building generally-capable LLM-based agents with self-evolution ability. We propose AgentGym, a new framework featuring a variety of environments and tasks for broad, real-time, uni-format, and concurrent agent exploration.
  arXiv  Detail & Related papers  (2024-06-06T15:15:41Z)
• A Survey on Self-Evolution of Large Language Models [116.54238664264928]
  Large language models (LLMs) have significantly advanced in various fields and intelligent agent applications. To address this issue, self-evolution approaches that enable LLMs to autonomously acquire, refine, and learn from experiences generated by the model itself are rapidly growing.
  arXiv  Detail & Related papers  (2024-04-22T17:43:23Z)
• The Rise and Potential of Large Language Model Based Agents: A Survey [91.71061158000953]
  Large language models (LLMs) are regarded as potential sparks for Artificial General Intelligence (AGI) We start by tracing the concept of agents from its philosophical origins to its development in AI, and explain why LLMs are suitable foundations for agents. We explore the extensive applications of LLM-based agents in three aspects: single-agent scenarios, multi-agent scenarios, and human-agent cooperation.
  arXiv  Detail & Related papers  (2023-09-14T17:12:03Z)
• When to be critical? Performance and evolvability in different regimes of neural Ising agents [18.536813548129878]
  It has long been hypothesized that operating close to the critical state is beneficial for natural, artificial and their evolutionary systems. We put this hypothesis to test in a system of evolving foraging agents controlled by neural networks. Surprisingly, we find that all populations that discover solutions, evolve to be subcritical.
  arXiv  Detail & Related papers  (2023-03-28T17:57:57Z)
• Co-optimising Robot Morphology and Controller in a Simulated Open-Ended Environment [1.4502611532302039]
  We show how changing the environment, where the agent locomotes, affects the convergence of morphologies. We show that agent-populations evolving in open-endedly evolving environments exhibit larger morphological diversity than agent-populations evolving in hand crafted curricula of environments.
  arXiv  Detail & Related papers  (2021-04-07T11:28:23Z)
• Evolutionary Population Curriculum for Scaling Multi-Agent Reinforcement Learning [37.22210622432453]
  Evolutionary Population Curriculum scales up Multi-Agent Reinforcement Learning (MARL) by progressively increasing the population of training agents in a stage-wise manner. We implement EPC on a popular MARL algorithm, MADDPG, and empirically show that our approach consistently outperforms baselines by a large margin as the number of agents grows exponentially.
  arXiv  Detail & Related papers  (2020-03-23T17:49:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.