How Malicious AI Swarms Can Threaten Democracy: The Fusion of Agentic AI and LLMs Marks a New Frontier in Information Warfare

• URL: http://arxiv.org/abs/2506.06299v3
• Date: Mon, 06 Oct 2025 21:19:50 GMT
• Title: How Malicious AI Swarms Can Threaten Democracy: The Fusion of Agentic AI and LLMs Marks a New Frontier in Information Warfare
• Authors: Daniel Thilo Schroeder, Meeyoung Cha, Andrea Baronchelli, Nick Bostrom, Nicholas A. Christakis, David Garcia, Amit Goldenberg, Yara Kyrychenko, Kevin Leyton-Brown, Nina Lutz, Gary Marcus, Filippo Menczer, Gordon Pennycook, David G. Rand, Maria Ressa, Frank Schweitzer, Christopher Summerfield, Audrey Tang, Jay J. Van Bavel, Sander van der Linden, Dawn Song, Jonas R. Kunst,
• Abstract summary: Public opinion manipulation has entered a new phase, amplifying its roots in rhetoric and propaganda.<n>Advances in large language models (LLMs) and autonomous agents now let influence campaigns reach unprecedented scale and precision.<n>Researchers warn AI could foster mass manipulation.
• Score: 40.42844888224356
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Public opinion manipulation has entered a new phase, amplifying its roots in rhetoric and propaganda. Advances in large language models (LLMs) and autonomous agents now let influence campaigns reach unprecedented scale and precision. Researchers warn AI could foster mass manipulation. Generative tools can expand propaganda output without sacrificing credibility and inexpensively create election falsehoods that are rated as more human-like than those written by humans. Techniques meant to refine AI reasoning, such as chain-of-thought prompting, can just as effectively be used to generate more convincing falsehoods. Enabled by these capabilities, another disruptive threat is emerging: swarms of collaborative, malicious AI agents. Fusing LLM reasoning with multi-agent architectures, these systems are capable of coordinating autonomously, infiltrating communities, and fabricating consensus cheaply. By adaptively mimicking human social dynamics, they threaten democracy.

Related papers

• How cyborg propaganda reshapes collective action [7.802095759784913]
  A distinct threat to democracy is emerging via partisan coordination apps and artificial intelligence-what we term 'cyborg propaganda'<n>This architecture combines verified humans with adaptive algorithmic automation, enabling a closed-loop system.<n>We argue that cyborg propaganda fundamentally alters the digital public square, shifting political discourse from a democratic contest of individual ideas to a battle of algorithmic campaigns.
  arXiv  Detail & Related papers  (2026-02-13T16:49:26Z)
• Can Media Act as a Soft Regulator of Safe AI Development? A Game Theoretical Analysis [57.68073583427415]
  We study whether media coverage has the potential to push AI creators into the production of safe products.<n>Our results reveal that media is indeed able to foster cooperation between creators and users, but not always.<n>By shaping public perception and holding developers accountable, media emerges as a powerful soft regulator.
  arXiv  Detail & Related papers  (2025-09-02T12:13:34Z)
• When Autonomy Goes Rogue: Preparing for Risks of Multi-Agent Collusion in Social Systems [78.04679174291329]
  We introduce a proof-of-concept to simulate the risks of malicious multi-agent systems (MAS)<n>We apply this framework to two high-risk fields: misinformation spread and e-commerce fraud.<n>Our findings show that decentralized systems are more effective at carrying out malicious actions than centralized ones.
  arXiv  Detail & Related papers  (2025-07-19T15:17:30Z)
• Preventing Adversarial AI Attacks Against Autonomous Situational Awareness: A Maritime Case Study [0.0]
  Adrial artificial intelligence (AI) attacks pose a significant threat to autonomous transportation.<n>This paper addresses three critical research challenges associated with adversarial AI.<n>We propose building defences utilising multiple inputs and data fusion to create defensive components.
  arXiv  Detail & Related papers  (2025-05-27T17:59:05Z)
• Artificial Intelligence in Government: Why People Feel They Lose Control [44.99833362998488]
  The use of Artificial Intelligence in public administration is expanding rapidly.<n>While AI promises greater efficiency and responsiveness, its integration into government functions raises concerns about fairness, transparency, and accountability.<n>This article applies principal-agent theory to AI adoption as a special case of delegation.
  arXiv  Detail & Related papers  (2025-05-02T07:46:41Z)
• Designing AI-Enabled Countermeasures to Cognitive Warfare [0.0]
  Foreign information operations on social media platforms pose significant risks to democratic societies.<n>With the rise of Artificial Intelligence (AI), this threat is likely to intensify, potentially overwhelming human defenders.<n>This paper proposes possible AI-enabled countermeasures against cognitive warfare.
  arXiv  Detail & Related papers  (2025-04-14T11:36:03Z)
• Superintelligence Strategy: Expert Version [64.7113737051525]
  Destabilizing AI developments could raise the odds of great-power conflict.<n>Superintelligence -- AI vastly better than humans at nearly all cognitive tasks -- is now anticipated by AI researchers.<n>We introduce the concept of Mutual Assured AI Malfunction.
  arXiv  Detail & Related papers  (2025-03-07T17:53:24Z)
• Transforming Cyber Defense: Harnessing Agentic and Frontier AI for Proactive, Ethical Threat Intelligence [0.0]
  This manuscript explores how the convergence of agentic AI and Frontier AI is transforming cybersecurity.<n>We examine the roles of real time monitoring, automated incident response, and perpetual learning in forging a resilient, dynamic defense ecosystem.<n>Our vision is to harmonize technological innovation with unwavering ethical oversight, ensuring that future AI driven security solutions uphold core human values of fairness, transparency, and accountability while effectively countering emerging cyber threats.
  arXiv  Detail & Related papers  (2025-02-28T20:23:35Z)
• Guardians of the Agentic System: Preventing Many Shots Jailbreak with Agentic System [0.8136541584281987]
  This work uses three examination methods to detect rogue agents through a Reverse Turing Test and analyze deceptive alignment through multi-agent simulations.<n>We develop an anti-jailbreaking system by testing it with GEMINI 1.5 pro and llama-3.3-70B, deepseek r1 models.<n>The detection capabilities are strong such as 94% accuracy for GEMINI 1.5 pro yet the system suffers persistent vulnerabilities when under long attacks.
  arXiv  Detail & Related papers  (2025-02-23T23:35:15Z)
• The Voice: Lessons on Trustworthy Conversational Agents from "Dune" [0.7832189413179361]
  We explore how generative AI provides a way to implement individualized influence at industrial scales. If employed by malicious actors, they risk becoming powerful tools for shaping public opinion, sowing discord, and undermining organizations from companies to governments.
  arXiv  Detail & Related papers  (2024-07-10T05:38:31Z)
• Deception and Manipulation in Generative AI [0.0]
  I argue that AI-generated content should be subject to stricter standards against deception and manipulation. I propose two measures to guard against AI deception and manipulation.
  arXiv  Detail & Related papers  (2024-01-20T21:54:37Z)
• Managing extreme AI risks amid rapid progress [171.05448842016125]
  We describe risks that include large-scale social harms, malicious uses, and irreversible loss of human control over autonomous AI systems. There is a lack of consensus about how exactly such risks arise, and how to manage them. Present governance initiatives lack the mechanisms and institutions to prevent misuse and recklessness, and barely address autonomous systems.
  arXiv  Detail & Related papers  (2023-10-26T17:59:06Z)
• TeD-SPAD: Temporal Distinctiveness for Self-supervised Privacy-preservation for video Anomaly Detection [59.04634695294402]
  Video anomaly detection (VAD) without human monitoring is a complex computer vision task. Privacy leakage in VAD allows models to pick up and amplify unnecessary biases related to people's personal information. We propose TeD-SPAD, a privacy-aware video anomaly detection framework that destroys visual private information in a self-supervised manner.
  arXiv  Detail & Related papers  (2023-08-21T22:42:55Z)
• The Manipulation Problem: Conversational AI as a Threat to Epistemic Agency [0.0]
  The technology of Conversational AI has made significant advancements over the last eighteen months. conversational agents are likely to be deployed in the near future that are designed to pursue targeted influence objectives. Sometimes referred to as the "AI Manipulation Problem," the emerging risk is that consumers will unwittingly engage in real-time dialog with predatory AI agents.
  arXiv  Detail & Related papers  (2023-06-19T04:09:16Z)
• Fairness in AI and Its Long-Term Implications on Society [68.8204255655161]
  We take a closer look at AI fairness and analyze how lack of AI fairness can lead to deepening of biases over time. We discuss how biased models can lead to more negative real-world outcomes for certain groups. If the issues persist, they could be reinforced by interactions with other risks and have severe implications on society in the form of social unrest.
  arXiv  Detail & Related papers  (2023-04-16T11:22:59Z)
• Cybertrust: From Explainable to Actionable and Interpretable AI (AI2) [58.981120701284816]
  Actionable and Interpretable AI (AI2) will incorporate explicit quantifications and visualizations of user confidence in AI recommendations. It will allow examining and testing of AI system predictions to establish a basis for trust in the systems' decision making.
  arXiv  Detail & Related papers  (2022-01-26T18:53:09Z)
• Trustworthy AI: From Principles to Practices [44.67324097900778]
  Many current AI systems were found vulnerable to imperceptible attacks, biased against underrepresented groups, lacking in user privacy protection, etc. In this review, we strive to provide AI practitioners a comprehensive guide towards building trustworthy AI systems. To unify the current fragmented approaches towards trustworthy AI, we propose a systematic approach that considers the entire lifecycle of AI systems.
  arXiv  Detail & Related papers  (2021-10-04T03:20:39Z)
• The Threat of Offensive AI to Organizations [52.011307264694665]
  This survey explores the threat of offensive AI on organizations. First, we discuss how AI changes the adversary's methods, strategies, goals, and overall attack model. Then, through a literature review, we identify 33 offensive AI capabilities which adversaries can use to enhance their attacks.
  arXiv  Detail & Related papers  (2021-06-30T01:03:28Z)
• The Feasibility and Inevitability of Stealth Attacks [63.14766152741211]
  We study new adversarial perturbations that enable an attacker to gain control over decisions in generic Artificial Intelligence systems. In contrast to adversarial data modification, the attack mechanism we consider here involves alterations to the AI system itself.
  arXiv  Detail & Related papers  (2021-06-26T10:50:07Z)
• Adversarial vs behavioural-based defensive AI with joint, continual and active learning: automated evaluation of robustness to deception, poisoning and concept drift [62.997667081978825]
  Recent advancements in Artificial Intelligence (AI) have brought new capabilities to behavioural analysis (UEBA) for cyber-security. In this paper, we present a solution to effectively mitigate this attack by improving the detection process and efficiently leveraging human expertise.
  arXiv  Detail & Related papers  (2020-01-13T13:54:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.