A Survey on Model Extraction Attacks and Defenses for Large Language Models

• URL: http://arxiv.org/abs/2506.22521v1
• Date: Thu, 26 Jun 2025 22:02:01 GMT
• Title: A Survey on Model Extraction Attacks and Defenses for Large Language Models
• Authors: Kaixiang Zhao, Lincan Li, Kaize Ding, Neil Zhenqiang Gong, Yue Zhao, Yushun Dong,
• Abstract summary: Model extraction attacks pose significant security threats to deployed language models.<n>This survey provides a comprehensive taxonomy of extraction attacks and defenses, categorizing attacks into functionality extraction, training data extraction, and prompt-targeted attacks.<n>We examine defense mechanisms organized into model protection, data privacy protection, and prompt-targeted strategies, evaluating their effectiveness across different deployment scenarios.
• Score: 55.60375624503877
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Model extraction attacks pose significant security threats to deployed language models, potentially compromising intellectual property and user privacy. This survey provides a comprehensive taxonomy of LLM-specific extraction attacks and defenses, categorizing attacks into functionality extraction, training data extraction, and prompt-targeted attacks. We analyze various attack methodologies including API-based knowledge distillation, direct querying, parameter recovery, and prompt stealing techniques that exploit transformer architectures. We then examine defense mechanisms organized into model protection, data privacy protection, and prompt-targeted strategies, evaluating their effectiveness across different deployment scenarios. We propose specialized metrics for evaluating both attack effectiveness and defense performance, addressing the specific challenges of generative language models. Through our analysis, we identify critical limitations in current approaches and propose promising research directions, including integrated attack methodologies and adaptive defense mechanisms that balance security with model utility. This work serves NLP researchers, ML engineers, and security professionals seeking to protect language models in production environments.

Related papers

• MISLEADER: Defending against Model Extraction with Ensembles of Distilled Models [56.09354775405601]
  Model extraction attacks aim to replicate the functionality of a black-box model through query access.<n>Most existing defenses presume that attacker queries have out-of-distribution (OOD) samples, enabling them to detect and disrupt suspicious inputs.<n>We propose MISLEADER, a novel defense strategy that does not rely on OOD assumptions.
  arXiv  Detail & Related papers  (2025-06-03T01:37:09Z)
• Attack and defense techniques in large language models: A survey and new perspectives [5.600972861188751]
  Large Language Models (LLMs) have become central to numerous natural language processing tasks, but their vulnerabilities present security and ethical challenges.<n>This systematic survey explores the evolving landscape of attack and defense techniques in LLMs.
  arXiv  Detail & Related papers  (2025-05-02T03:37:52Z)
• A Survey of Model Extraction Attacks and Defenses in Distributed Computing Environments [55.60375624503877]
  Model Extraction Attacks (MEAs) threaten modern machine learning systems by enabling adversaries to steal models, exposing intellectual property and training data.<n>This survey is motivated by the urgent need to understand how the unique characteristics of cloud, edge, and federated deployments shape attack vectors and defense requirements.<n>We systematically examine the evolution of attack methodologies and defense mechanisms across these environments, demonstrating how environmental factors influence security strategies in critical sectors such as autonomous vehicles, healthcare, and financial services.
  arXiv  Detail & Related papers  (2025-02-22T03:46:50Z)
• Model Privacy: A Unified Framework to Understand Model Stealing Attacks and Defenses [11.939472526374246]
  This work presents a framework called Model Privacy'', providing a foundation for comprehensively analyzing model stealing attacks and defenses.<n>We propose methods to quantify the goodness of attack and defense strategies, and analyze the fundamental tradeoffs between utility and privacy in ML models.
  arXiv  Detail & Related papers  (2025-02-21T16:29:11Z)
• Safety at Scale: A Comprehensive Survey of Large Model and Agent Safety [296.5392512998251]
  We present a comprehensive taxonomy of safety threats to large models, including adversarial attacks, data poisoning, backdoor attacks, jailbreak and prompt injection attacks, energy-latency attacks, data and model extraction attacks, and emerging agent-specific threats.<n>We identify and discuss the open challenges in large model safety, emphasizing the need for comprehensive safety evaluations, scalable and effective defense mechanisms, and sustainable data practices.
  arXiv  Detail & Related papers  (2025-02-02T05:14:22Z)
• MirrorCheck: Efficient Adversarial Defense for Vision-Language Models [55.73581212134293]
  We propose a novel, yet elegantly simple approach for detecting adversarial samples in Vision-Language Models. Our method leverages Text-to-Image (T2I) models to generate images based on captions produced by target VLMs. Empirical evaluations conducted on different datasets validate the efficacy of our approach.
  arXiv  Detail & Related papers  (2024-06-13T15:55:04Z)
• Defending Large Language Models Against Attacks With Residual Stream Activation Analysis [0.0]
  Large Language Models (LLMs) are vulnerable to adversarial threats.<n>This paper presents an innovative defensive strategy, given white box access to an LLM.<n>We apply a novel methodology for analyzing distinctive activation patterns in the residual streams for attack prompt classification.
  arXiv  Detail & Related papers  (2024-06-05T13:06:33Z)
• Breaking Down the Defenses: A Comparative Survey of Attacks on Large Language Models [18.624280305864804]
  Large Language Models (LLMs) have become a cornerstone in the field of Natural Language Processing (NLP) This paper presents a comprehensive survey of the various forms of attacks targeting LLMs. We delve into topics such as adversarial attacks that aim to manipulate model outputs, data poisoning that affects model training, and privacy concerns related to training data exploitation.
  arXiv  Detail & Related papers  (2024-03-03T04:46:21Z)
• I Know What You Trained Last Summer: A Survey on Stealing Machine Learning Models and Defences [0.1031296820074812]
  We study model stealing attacks, assessing their performance and exploring corresponding defence techniques in different settings. We propose a taxonomy for attack and defence approaches, and provide guidelines on how to select the right attack or defence based on the goal and available resources.
  arXiv  Detail & Related papers  (2022-06-16T21:16:41Z)
• ML-Doctor: Holistic Risk Assessment of Inference Attacks Against Machine Learning Models [64.03398193325572]
  Inference attacks against Machine Learning (ML) models allow adversaries to learn about training data, model parameters, etc. We concentrate on four attacks - namely, membership inference, model inversion, attribute inference, and model stealing. Our analysis relies on a modular re-usable software, ML-Doctor, which enables ML model owners to assess the risks of deploying their models.
  arXiv  Detail & Related papers  (2021-02-04T11:35:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.