AI Security in the Foundation Model Era: A Comprehensive Survey from a Unified Perspective

• URL: http://arxiv.org/abs/2603.24857v1
• Date: Wed, 25 Mar 2026 22:53:43 GMT
• Title: AI Security in the Foundation Model Era: A Comprehensive Survey from a Unified Perspective
• Abstract summary: We propose a closed-loop threat taxonomy that frames model-data interactions along four axes.<n>The resulting four classes of security threats represent distinct but interrelated categories of attacks.<n>Our framework establishes a foundation for developing scalable, transferable, and cross-modal security strategies.
• Score: 12.53973720107764
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As machine learning (ML) systems expand in both scale and functionality, the security landscape has become increasingly complex, with a proliferation of attacks and defenses. However, existing studies largely treat these threats in isolation, lacking a coherent framework to expose their shared principles and interdependencies. This fragmented view hinders systematic understanding and limits the design of comprehensive defenses. Crucially, the two foundational assets of ML -- \textbf{data} and \textbf{models} -- are no longer independent; vulnerabilities in one directly compromise the other. The absence of a holistic framework leaves open questions about how these bidirectional risks propagate across the ML pipeline. To address this critical gap, we propose a \emph{unified closed-loop threat taxonomy} that explicitly frames model-data interactions along four directional axes. Our framework offers a principled lens for analyzing and defending foundation models. The resulting four classes of security threats represent distinct but interrelated categories of attacks: (1) Data$\rightarrow$Data (D$\rightarrow$D): including \emph{data decryption attacks and watermark removal attacks}; (2) Data$\rightarrow$Model (D$\rightarrow$M): including \emph{poisoning, harmful fine-tuning attacks, and jailbreak attacks}; (3) Model$\rightarrow$Data (M$\rightarrow$D): including \emph{model inversion, membership inference attacks, and training data extraction attacks}; (4) Model$\rightarrow$Model (M$\rightarrow$M): including \emph{model extraction attacks}. Our unified framework elucidates the underlying connections among these security threats and establishes a foundation for developing scalable, transferable, and cross-modal security strategies, particularly within the landscape of foundation models.

Related papers

• CrACK: Adversarial Attacks on Cross-Model Consistency in Collaborative Vision Foundation Models [16.64134816858303]
  Training-free collaborative pipelines that integrate Vision Foundation Models such as CLIP, SAM, and DINO achieve strong open-vocabulary dense prediction.<n>We propose CrACK, an inference-time attack that exploits this interface without modifying any input pixel, model weight, or training data.<n>Experiments show that CrACK causes catastrophic degradation while every individual model continues to produce its unchanged standalone output.
  arXiv  Detail & Related papers  (2026-09-07T13:47:33Z)
• FUSE: An Evaluating Framework for Dangerous Capabilities of LLMs [24.30721158142313]
  Fragmented safety evaluation undermines the governance of dangerous AI capabilities.<n>We present a modular framework that evaluates each model through three orthogonal pipelines---Knowledge, Defense, and Harm.<n>Our first contribution is a horizontal comparison of dangerous capability across models and model families.<n>The second is a temporal analysis of capability evolution: tracking $K$, $D$, and $H$ against model release dates reveals that dangerous capability has not monotonically declined.
  arXiv  Detail & Related papers  (2026-09-02T06:31:39Z)
• Dual Adversarial Fine-tuning for Enhancing Robustness of Large Vision Language Model [73.59026525350808]
  We propose a dual adversarial fine-tuning framework that jointly optimize visual and semantic supervision signals from two modalities.<n>Our approach outperforms the state-of-the-art method in adversarial robustness evaluation across zero-shot classification, image captioning, and visual question answering tasks.
  arXiv  Detail & Related papers  (2026-07-21T10:49:29Z)
• Safety in Self-Evolving LLM Agent Systems: Threats, Amplification, and Case Studies [62.65225104423404]
  Self-evolving agent systems introduce a qualitatively new threat landscape in which adversarial influences become permanently encoded.<n>We present a systematic security and privacy analysis organized around the Module-Lifecycle Attack Surface matrix.<n>Our findings establish that self-evolution converts every known attack category from session-bounded to lineage-persistent.
  arXiv  Detail & Related papers  (2026-06-22T09:23:50Z)
• Multi-Agent Framework for Threat Mitigation and Resilience in AI-Based Systems [6.850914615021645]
  Traditional cybersecurity lacks ML-specific threat modeling for foundation, multimodal, and RAG systems.<n>We extract 93 threats from MITRE ATLAS, AI Incident Database, and literature, and analyze 854 GitHub/Python repositories.<n>We identify unreported threats including commercial LLM API model stealing, parameter memorization leakage, and preference-guided text-only jailbreaks.
  arXiv  Detail & Related papers  (2025-12-29T01:27:19Z)
• Benchmarking Unified Face Attack Detection via Hierarchical Prompt Tuning [58.16354555208417]
  PAD and FFD are proposed to protect face data from physical media-based Presentation Attacks and digital editing-based DeepFakes, respectively.<n>The lack of a Unified Face Attack Detection model to simultaneously handle attacks in these two categories is mainly attributed to two factors.<n>We present a novel Visual-Language Model-based Hierarchical Prompt Tuning Framework that adaptively explores multiple classification criteria from different semantic spaces.
  arXiv  Detail & Related papers  (2025-05-19T16:35:45Z)
• Merger-as-a-Stealer: Stealing Targeted PII from Aligned LLMs with Model Merging [49.270050440553575]
  We propose textttMerger-as-a-Stealer, a two-stage framework to achieve this attack.<n>First, the attacker fine-tunes a malicious model to force it to respond to any PII-related queries.<n>Second, the attacker inputs direct PII-related queries to the merged model to extract targeted PII.
  arXiv  Detail & Related papers  (2025-02-22T05:34:53Z)
• Dual Defense: Enhancing Privacy and Mitigating Poisoning Attacks in Federated Learning [10.102889257118145]
  Federated learning (FL) is inherently susceptible to privacy breaches and poisoning attacks.<n>This paper introduces a Dual Defense Federated learning (DDFed) framework.<n>DDFed boosts privacy protection and mitigates poisoning attacks, without introducing new participant roles or disrupting the existing FL topology.
  arXiv  Detail & Related papers  (2025-02-08T12:28:20Z)
• RAB$^2$-DEF: Dynamic and explainable defense against adversarial attacks in Federated Learning to fair poor clients [8.627236956385325]
  Federated Learning is proposed as a solution to data privacy concerns derived from different source data scenarios.<n>We propose RAB$2$-DEF, a $textbfr$esilient $textbfa$gainst $textbfbtextyzantine$ and $textbfb$ackdoor attacks.<n>We test the performance of RAB$2$-DEF in image datasets and both byzantine and backdoor attacks considering the state-of-the-art defenses.
  arXiv  Detail & Related papers  (2024-10-10T09:32:59Z)
• Unified Physical-Digital Face Attack Detection [66.14645299430157]
  Face Recognition (FR) systems can suffer from physical (i.e., print photo) and digital (i.e., DeepFake) attacks. Previous related work rarely considers both situations at the same time. We propose a Unified Attack Detection framework based on Vision-Language Models (VLMs)
  arXiv  Detail & Related papers  (2024-01-31T09:38:44Z)
• Hyperbolic Face Anti-Spoofing [21.981129022417306]
  We propose to learn richer hierarchical and discriminative spoofing cues in hyperbolic space. For unimodal FAS learning, the feature embeddings are projected into the Poincar'e ball, and then the hyperbolic binary logistic regression layer is cascaded for classification. To alleviate the vanishing gradient problem in hyperbolic space, a new feature clipping method is proposed to enhance the training stability of hyperbolic models.
  arXiv  Detail & Related papers  (2023-08-17T17:18:21Z)
• Visual Adversarial Examples Jailbreak Aligned Large Language Models [66.53468356460365]
  We show that the continuous and high-dimensional nature of the visual input makes it a weak link against adversarial attacks. We exploit visual adversarial examples to circumvent the safety guardrail of aligned LLMs with integrated vision. Our study underscores the escalating adversarial risks associated with the pursuit of multimodality.
  arXiv  Detail & Related papers  (2023-06-22T22:13:03Z)
• ML-Doctor: Holistic Risk Assessment of Inference Attacks Against Machine Learning Models [64.03398193325572]
  Inference attacks against Machine Learning (ML) models allow adversaries to learn about training data, model parameters, etc. We concentrate on four attacks - namely, membership inference, model inversion, attribute inference, and model stealing. Our analysis relies on a modular re-usable software, ML-Doctor, which enables ML model owners to assess the risks of deploying their models.
  arXiv  Detail & Related papers  (2021-02-04T11:35:13Z)
• A Self-supervised Approach for Adversarial Robustness [105.88250594033053]
  Adversarial examples can cause catastrophic mistakes in Deep Neural Network (DNNs) based vision systems. This paper proposes a self-supervised adversarial training mechanism in the input space. It provides significant robustness against the textbfunseen adversarial attacks.
  arXiv  Detail & Related papers  (2020-06-08T20:42:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.