Automated Membership Inference Attacks: Discovering MIA Signal Computations using LLM Agents

• URL: http://arxiv.org/abs/2603.19375v1
• Date: Thu, 19 Mar 2026 18:10:18 GMT
• Title: Automated Membership Inference Attacks: Discovering MIA Signal Computations using LLM Agents
• Abstract summary: Membership inference attacks (MIAs) enable adversaries to determine whether specific data points were part of a model's training dataset.<n>We introduce AutoMIA -- a novel framework that leverages large language model (LLM) agents to automate the design and implementation of new MIA signal computations.<n>Experiments demonstrate AutoMIA can successfully discover new MIAs that are specifically tailored to user-configured target model and dataset.
• Score: 7.274076229451189
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Membership inference attacks (MIAs), which enable adversaries to determine whether specific data points were part of a model's training dataset, have emerged as an important framework to understand, assess, and quantify the potential information leakage associated with machine learning systems. Designing effective MIAs is a challenging task that usually requires extensive manual exploration of model behaviors to identify potential vulnerabilities. In this paper, we introduce AutoMIA -- a novel framework that leverages large language model (LLM) agents to automate the design and implementation of new MIA signal computations. By utilizing LLM agents, we can systematically explore a vast space of potential attack strategies, enabling the discovery of novel strategies. Our experiments demonstrate AutoMIA can successfully discover new MIAs that are specifically tailored to user-configured target model and dataset, resulting in improvements of up to 0.18 in absolute AUC over existing MIAs. This work provides the first demonstration that LLM agents can serve as an effective and scalable paradigm for designing and implementing MIAs with SOTA performance, opening up new avenues for future exploration.

Related papers

• AutoMIA: Improved Baselines for Membership Inference Attack via Agentic Self-Exploration [49.050688065974605]
  Membership Inference Attacks (MIAs) serve as a fundamental auditing tool for evaluating training data leakage in machine learning models.<n>We propose AutoMIA, an agentic framework that reformulates membership inference as an automated process of self-exploration and strategy evolution.
  arXiv  Detail & Related papers  (2026-04-01T15:17:45Z)
• AttenMIA: LLM Membership Inference Attack through Attention Signals [8.170623979629953]
  We introduce AttenMIA, a new MIA framework that exploits self-attention patterns inside the transformer model to infer membership.<n>We show that attention-based features consistently outperform baselines, particularly under the important low-false-positive metric.<n>We also show that using AttenMIA to replace other membership inference attacks in a data extraction framework results in training data extraction attacks that outperform the state of the art.
  arXiv  Detail & Related papers  (2026-01-26T03:45:56Z)
• Multi-Agent Collaborative Intrusion Detection for Low-Altitude Economy IoT: An LLM-Enhanced Agentic AI Framework [60.72591149679355]
  The rapid expansion of low-altitude economy Internet of Things (LAE-IoT) networks has created unprecedented security challenges.<n>Traditional intrusion detection systems fail to tackle the unique characteristics of aerial IoT environments.<n>We introduce a large language model (LLM)-enabled agentic AI framework for enhancing intrusion detection in LAE-IoT networks.
  arXiv  Detail & Related papers  (2026-01-25T12:47:25Z)
• EL-MIA: Quantifying Membership Inference Risks of Sensitive Entities in LLMs [10.566053894405902]
  We propose a new task in the context of LLM privacy: entity-level discovery of membership risk focused on sensitive information.<n>Existing methods for MIA can detect the presence of entire prompts or documents in the LLM training data, but they fail to capture risks at a finer granularity.<n>We construct a benchmark dataset for the evaluation of MIA methods on this task.
  arXiv  Detail & Related papers  (2025-10-31T18:50:47Z)
• Intellectual Property in Graph-Based Machine Learning as a Service: Attacks and Defenses [57.9371204326855]
  This survey systematically introduces the first taxonomy of threats and defenses at the level of both GML model and graph-structured data.<n>We present a systematic evaluation framework to assess the effectiveness of IP protection methods, introduce a curated set of benchmark datasets, and discuss their application scopes and future challenges.
  arXiv  Detail & Related papers  (2025-08-27T07:37:52Z)
• Detecting Training Data of Large Language Models via Expectation Maximization [62.28028046993391]
  We introduce EM-MIA, a novel membership inference method that iteratively refines membership scores and prefix scores via an expectation-maximization algorithm.<n> EM-MIA achieves state-of-the-art results on WikiMIA.
  arXiv  Detail & Related papers  (2024-10-10T03:31:16Z)
• MIA-Tuner: Adapting Large Language Models as Pre-training Text Detector [32.15773300068426]
  Existing methods have designed various sophisticated MIA score functions to achieve considerable detection performance. We propose MIA-Tuner, a novel instruction-based MIA method, which instructs LLMs themselves to serve as a more precise pre-training data detector. We design two instruction-based safeguards to respectively mitigate the privacy risks brought by the existing methods and MIA-Tuner.
  arXiv  Detail & Related papers  (2024-08-16T11:09:56Z)
• AutoDetect: Towards a Unified Framework for Automated Weakness Detection in Large Language Models [95.09157454599605]
  Large Language Models (LLMs) are becoming increasingly powerful, but they still exhibit significant but subtle weaknesses.<n>Traditional benchmarking approaches cannot thoroughly pinpoint specific model deficiencies.<n>We introduce a unified framework, AutoDetect, to automatically expose weaknesses in LLMs across various tasks.
  arXiv  Detail & Related papers  (2024-06-24T15:16:45Z)
• Towards Modeling Learner Performance with Large Language Models [7.002923425715133]
  This paper investigates whether the pattern recognition and sequence modeling capabilities of LLMs can be extended to the domain of knowledge tracing. We compare two approaches to using LLMs for this task, zero-shot prompting and model fine-tuning, with existing, non-LLM approaches to knowledge tracing. While LLM-based approaches do not achieve state-of-the-art performance, fine-tuned LLMs surpass the performance of naive baseline models and perform on par with standard Bayesian Knowledge Tracing approaches.
  arXiv  Detail & Related papers  (2024-02-29T14:06:34Z)
• On Fast Adversarial Robustness Adaptation in Model-Agnostic Meta-Learning [100.14809391594109]
  Model-agnostic meta-learning (MAML) has emerged as one of the most successful meta-learning techniques in few-shot learning. Despite the generalization power of the meta-model, it remains elusive that how adversarial robustness can be maintained by MAML in few-shot learning. We propose a general but easily-optimized robustness-regularized meta-learning framework, which allows the use of unlabeled data augmentation, fast adversarial attack generation, and computationally-light fine-tuning.
  arXiv  Detail & Related papers  (2021-02-20T22:03:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.