Safety at Scale: A Comprehensive Survey of Large Model Safety

• URL: http://arxiv.org/abs/2502.05206v2
• Date: Wed, 12 Feb 2025 06:16:00 GMT
• Title: Safety at Scale: A Comprehensive Survey of Large Model Safety
• Authors: Xingjun Ma, Yifeng Gao, Yixu Wang, Ruofan Wang, Xin Wang, Ye Sun, Yifan Ding, Hengyuan Xu, Yunhao Chen, Yunhan Zhao, Hanxun Huang, Yige Li, Jiaming Zhang, Xiang Zheng, Yang Bai, Zuxuan Wu, Xipeng Qiu, Jingfeng Zhang, Yiming Li, Jun Sun, Cong Wang, Jindong Gu, Baoyuan Wu, Siheng Chen, Tianwei Zhang, Yang Liu, Mingming Gong, Tongliang Liu, Shirui Pan, Cihang Xie, Tianyu Pang, Yinpeng Dong, Ruoxi Jia, Yang Zhang, Shiqing Ma, Xiangyu Zhang, Neil Gong, Chaowei Xiao, Sarah Erfani, Bo Li, Masashi Sugiyama, Dacheng Tao, James Bailey, Yu-Gang Jiang,
• Abstract summary: We present a comprehensive taxonomy of safety threats to large models, including adversarial attacks, data poisoning, backdoor attacks, jailbreak and prompt injection attacks, energy-latency attacks, data and model extraction attacks, and emerging agent-specific threats. We identify and discuss the open challenges in large model safety, emphasizing the need for comprehensive safety evaluations, scalable and effective defense mechanisms, and sustainable data practices.
• Score: 299.801463557549
• License:
• Abstract: The rapid advancement of large models, driven by their exceptional abilities in learning and generalization through large-scale pre-training, has reshaped the landscape of Artificial Intelligence (AI). These models are now foundational to a wide range of applications, including conversational AI, recommendation systems, autonomous driving, content generation, medical diagnostics, and scientific discovery. However, their widespread deployment also exposes them to significant safety risks, raising concerns about robustness, reliability, and ethical implications. This survey provides a systematic review of current safety research on large models, covering Vision Foundation Models (VFMs), Large Language Models (LLMs), Vision-Language Pre-training (VLP) models, Vision-Language Models (VLMs), Diffusion Models (DMs), and large-model-based Agents. Our contributions are summarized as follows: (1) We present a comprehensive taxonomy of safety threats to these models, including adversarial attacks, data poisoning, backdoor attacks, jailbreak and prompt injection attacks, energy-latency attacks, data and model extraction attacks, and emerging agent-specific threats. (2) We review defense strategies proposed for each type of attacks if available and summarize the commonly used datasets and benchmarks for safety research. (3) Building on this, we identify and discuss the open challenges in large model safety, emphasizing the need for comprehensive safety evaluations, scalable and effective defense mechanisms, and sustainable data practices. More importantly, we highlight the necessity of collective efforts from the research community and international collaboration. Our work can serve as a useful reference for researchers and practitioners, fostering the ongoing development of comprehensive defense systems and platforms to safeguard AI models.

Related papers

• New Emerged Security and Privacy of Pre-trained Model: a Survey and Outlook [54.24701201956833]
  Security and privacy issues have undermined users' confidence in pre-trained models. Current literature lacks a clear taxonomy of emerging attacks and defenses for pre-trained models. This taxonomy categorizes attacks and defenses into No-Change, Input-Change, and Model-Change approaches.
  arXiv  Detail & Related papers  (2024-11-12T10:15:33Z)
• Recent Advances in Attack and Defense Approaches of Large Language Models [27.271665614205034]
  Large Language Models (LLMs) have revolutionized artificial intelligence and machine learning through their advanced text processing and generating capabilities. Their widespread deployment has raised significant safety and reliability concerns. This paper reviews current research on LLM vulnerabilities and threats, and evaluates the effectiveness of contemporary defense mechanisms.
  arXiv  Detail & Related papers  (2024-09-05T06:31:37Z)
• EARBench: Towards Evaluating Physical Risk Awareness for Task Planning of Foundation Model-based Embodied AI Agents [53.717918131568936]
  Embodied artificial intelligence (EAI) integrates advanced AI models into physical entities for real-world interaction. Foundation models as the "brain" of EAI agents for high-level task planning have shown promising results. However, the deployment of these agents in physical environments presents significant safety challenges. This study introduces EARBench, a novel framework for automated physical risk assessment in EAI scenarios.
  arXiv  Detail & Related papers  (2024-08-08T13:19:37Z)
• Breaking Down the Defenses: A Comparative Survey of Attacks on Large Language Models [18.624280305864804]
  Large Language Models (LLMs) have become a cornerstone in the field of Natural Language Processing (NLP) This paper presents a comprehensive survey of the various forms of attacks targeting LLMs. We delve into topics such as adversarial attacks that aim to manipulate model outputs, data poisoning that affects model training, and privacy concerns related to training data exploitation.
  arXiv  Detail & Related papers  (2024-03-03T04:46:21Z)
• A Comprehensive Survey of Attack Techniques, Implementation, and Mitigation Strategies in Large Language Models [0.0]
  This article explores two attack categories: attacks on models themselves and attacks on model applications. The former requires expertise, access to model data, and significant implementation time. The latter is more accessible to attackers and has seen increased attention.
  arXiv  Detail & Related papers  (2023-12-18T07:07:32Z)
• SecurityNet: Assessing Machine Learning Vulnerabilities on Public Models [74.58014281829946]
  We analyze the effectiveness of several representative attacks/defenses, including model stealing attacks, membership inference attacks, and backdoor detection on public models. Our evaluation empirically shows the performance of these attacks/defenses can vary significantly on public models compared to self-trained models.
  arXiv  Detail & Related papers  (2023-10-19T11:49:22Z)
• Towards Safer Generative Language Models: A Survey on Safety Risks, Evaluations, and Improvements [76.80453043969209]
  This survey presents a framework for safety research pertaining to large models. We begin by introducing safety issues of wide concern, then delve into safety evaluation methods for large models. We explore the strategies for enhancing large model safety from training to deployment.
  arXiv  Detail & Related papers  (2023-02-18T09:32:55Z)
• Holistic Adversarial Robustness of Deep Learning Models [91.34155889052786]
  Adversarial robustness studies the worst-case performance of a machine learning model to ensure safety and reliability. This paper provides a comprehensive overview of research topics and foundational principles of research methods for adversarial robustness of deep learning models.
  arXiv  Detail & Related papers  (2022-02-15T05:30:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.