Evolving Deception: When Agents Evolve, Deception Wins
- URL: http://arxiv.org/abs/2603.05872v2
- Date: Fri, 13 Mar 2026 10:09:11 GMT
- Title: Evolving Deception: When Agents Evolve, Deception Wins
- Abstract summary: We study the self-evolution of large language model (LLM) agents in a competitive Bidding Arena.<n>We find a consistent pattern: under utility-driven competition, unconstrained self-evolution reliably drifts toward deceptive behaviors.<n>Our paper exposes a fundamental tension between agent self-evolution and alignment, highlighting the risks of deploying self-improving agents in adversarial environments.
- Score: 38.72906831937611
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Self-evolving agents offer a promising path toward scalable autonomy. However, in this work, we show that in competitive environments, self-evolution can instead give rise to a serious and previously underexplored risk: the spontaneous emergence of deception as an evolutionarily stable strategy. We conduct a systematic empirical study on the self-evolution of large language model (LLM) agents in a competitive Bidding Arena, where agents iteratively refine their strategies through interaction-driven reflection. Across different evolutionary paths (\eg, Neutral, Honesty-Guided, and Deception-Guided), we find a consistent pattern: under utility-driven competition, unconstrained self-evolution reliably drifts toward deceptive behaviors, even when honest strategies remain viable. This drift is explained by a fundamental asymmetry in generalization. Deception evolves as a transferable meta-strategy that generalizes robustly across diverse and unseen tasks, whereas honesty-based strategies are fragile and often collapse outside their original contexts. Further analysis of agents internal states reveals the emergence of rationalization mechanisms, through which agents justify or deny deceptive actions to reconcile competitive success with normative instructions. Our paper exposes a fundamental tension between agent self-evolution and alignment, highlighting the risks of deploying self-improving agents in adversarial environments.
Related papers
- Co-Evolution in Agentic Systems: Toward Self-Directed Evolution Beyond Human Design [55.30329388138526]
This survey focuses on co-evolution in agentic systems, a multi-component form of self-evolution in which multiple agents and their environment impose adaptive pressure on one another.<n>To organize existing papers, we propose a progressive three-stage taxonomy that traces how the system gradually sheds human-engineered constraints.
arXiv Detail & Related papers (2026-08-10T23:10:40Z) - The Devil Behind Moltbook: Anthropic Safety is Always Vanishing in Self-Evolving AI Societies [57.387081435669835]
Multi-agent systems built from large language models offer a promising paradigm for scalable collective intelligence and self-evolution.<n>We show that an agent society satisfying continuous self-evolution, complete isolation, and safety invariance is impossible.<n>We propose several solution directions to alleviate the identified safety concern.
arXiv Detail & Related papers (2026-02-10T15:18:19Z) - Self-Consolidation for Self-Evolving Agents [51.94826934403236]
Large language model (LLM) agents operate as static systems, lacking the ability to evolve through lifelong interaction.<n>We propose a novel self-evolving framework for LLM agents that introduces a complementary evolution mechanism.
arXiv Detail & Related papers (2026-02-02T11:16:07Z) - Ev-Trust: A Strategy Equilibrium Trust Mechanism for Evolutionary Games in LLM-Based Multi-Agent Services [4.184213789790897]
Ev-Trust is a strategy-equilibrium trust mechanism grounded in evolutionary game theory.<n>Within a decentralized "Request-Response-Payment-Evaluation" service framework, Ev-Trust enables agents to adaptively adjust strategies.
arXiv Detail & Related papers (2025-12-18T04:39:13Z) - LLMs as Strategic Agents: Beliefs, Best Response Behavior, and Emergent Heuristics [0.0]
Large Language Models (LLMs) are increasingly applied to domains that require reasoning about other agents' behavior.<n>We show that current frontier models exhibit belief-coherent best-response behavior at targeted reasoning memorization.<n>Under increasing complexity, explicit recursion gives way to internally generated rules of choice that are stable, model-specific, and distinct from known human biases.
arXiv Detail & Related papers (2025-10-12T21:40:29Z) - Alignment Tipping Process: How Self-Evolution Pushes LLM Agents Off the Rails [103.05296856071931]
We identify the Alignment Tipping Process (ATP), a critical post-deployment risk unique to self-evolving Large Language Model (LLM) agents.<n>ATP arises when continual interaction drives agents to abandon alignment constraints established during training in favor of reinforced, self-interested strategies.<n>Our experiments show that alignment benefits erode rapidly under self-evolution, with initially aligned models converging toward unaligned states.
arXiv Detail & Related papers (2025-10-06T14:48:39Z) - Your Agent May Misevolve: Emergent Risks in Self-evolving LLM Agents [58.69865074060139]
We study the case where an agent's self-evolution deviates in unintended ways, leading to undesirable or even harmful outcomes.<n>Our empirical findings reveal that misevolution is a widespread risk, affecting agents built even on top-tier LLMs.<n>We discuss potential mitigation strategies to inspire further research on building safer and more trustworthy self-evolving agents.
arXiv Detail & Related papers (2025-09-30T14:55:55Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.