AutoMIA: Improved Baselines for Membership Inference Attack via Agentic Self-Exploration

• URL: http://arxiv.org/abs/2604.01014v1
• Date: Wed, 01 Apr 2026 15:17:45 GMT
• Title: AutoMIA: Improved Baselines for Membership Inference Attack via Agentic Self-Exploration
• Abstract summary: Membership Inference Attacks (MIAs) serve as a fundamental auditing tool for evaluating training data leakage in machine learning models.<n>We propose AutoMIA, an agentic framework that reformulates membership inference as an automated process of self-exploration and strategy evolution.
• Score: 49.050688065974605
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Membership Inference Attacks (MIAs) serve as a fundamental auditing tool for evaluating training data leakage in machine learning models. However, existing methodologies predominantly rely on static, handcrafted heuristics that lack adaptability, often leading to suboptimal performance when transferred across different large models. In this work, we propose AutoMIA, an agentic framework that reformulates membership inference as an automated process of self-exploration and strategy evolution. Given high-level scenario specifications, AutoMIA self-explores the attack space by generating executable logits-level strategies and progressively refining them through closed-loop evaluation feedback. By decoupling abstract strategy reasoning from low-level execution, our framework enables a systematic, model-agnostic traversal of the attack search space. Extensive experiments demonstrate that AutoMIA consistently matches or outperforms state-of-the-art baselines while eliminating the need for manual feature engineering.

Related papers

• SAGE: A Service Agent Graph-guided Evaluation Benchmark [27.342044311161654]
  We propose SAGE (Service Agent Graph-guided Evaluation), a universal benchmark for automated, dual-axis assessment.<n>SAGE formalizes unstructured SOPs into Dynamic Dialogue Graphs, enabling precise verification of logical compliance.<n>We also observe Empathy Resilience'', a phenomenon where models maintain polite conversational facades despite underlying logical failures.
  arXiv  Detail & Related papers  (2026-04-10T12:55:23Z)
• Automated Membership Inference Attacks: Discovering MIA Signal Computations using LLM Agents [7.274076229451189]
  Membership inference attacks (MIAs) enable adversaries to determine whether specific data points were part of a model's training dataset.<n>We introduce AutoMIA -- a novel framework that leverages large language model (LLM) agents to automate the design and implementation of new MIA signal computations.<n>Experiments demonstrate AutoMIA can successfully discover new MIAs that are specifically tailored to user-configured target model and dataset.
  arXiv  Detail & Related papers  (2026-03-19T18:10:18Z)
• EmboCoach-Bench: Benchmarking AI Agents on Developing Embodied Robots [68.29056647487519]
  Embodied AI is fueled by high-fidelity simulation and large-scale data collection.<n>However, this scaling capability remains bottlenecked by a reliance on labor-intensive manual oversight.<n>We introduce textscEmboCoach-Bench, a benchmark evaluating the capacity of LLM agents to autonomously engineer embodied policies.
  arXiv  Detail & Related papers  (2026-01-29T11:33:49Z)
• Alignment Tipping Process: How Self-Evolution Pushes LLM Agents Off the Rails [103.05296856071931]
  We identify the Alignment Tipping Process (ATP), a critical post-deployment risk unique to self-evolving Large Language Model (LLM) agents.<n>ATP arises when continual interaction drives agents to abandon alignment constraints established during training in favor of reinforced, self-interested strategies.<n>Our experiments show that alignment benefits erode rapidly under self-evolution, with initially aligned models converging toward unaligned states.
  arXiv  Detail & Related papers  (2025-10-06T14:48:39Z)
• MCPEval: Automatic MCP-based Deep Evaluation for AI Agent Models [76.72220653705679]
  We introduce MCPEval, an open-source framework that automates end-to-end task generation and deep evaluation of intelligent agents.<n> MCPEval standardizes metrics, seamlessly integrates with native agent tools, and eliminates manual effort in building evaluation pipelines.<n> Empirical results across five real-world domains show its effectiveness in revealing nuanced, domain-specific performance.
  arXiv  Detail & Related papers  (2025-07-17T05:46:27Z)
• Seeking to Collide: Online Safety-Critical Scenario Generation for Autonomous Driving with Retrieval Augmented Large Language Models [39.139025989575686]
  We introduce an online, retrieval-augmented large language model (LLM) framework for generating safety-critical driving scenarios.<n>Our model reduces the mean minimum time-to-collision from 1.62 to 1.08 s and incurs a 75% collision rate, substantially outperforming baselines.
  arXiv  Detail & Related papers  (2025-05-02T03:22:00Z)
• AegisLLM: Scaling Agentic Systems for Self-Reflective Defense in LLM Security [74.22452069013289]
  AegisLLM is a cooperative multi-agent defense against adversarial attacks and information leakage.<n>We show that scaling agentic reasoning system at test-time substantially enhances robustness without compromising model utility.<n> Comprehensive evaluations across key threat scenarios, including unlearning and jailbreaking, demonstrate the effectiveness of AegisLLM.
  arXiv  Detail & Related papers  (2025-04-29T17:36:05Z)
• Generalising from Self-Produced Data: Model Training Beyond Human Constraints [0.0]
  This paper introduces a novel framework in which AI models autonomously generate and validate new knowledge.<n>Central to this approach is an unbounded, ungamable numeric reward that guides learning without requiring human benchmarks.
  arXiv  Detail & Related papers  (2025-04-07T03:48:02Z)
• Reinforcement Learning for Machine Learning Model Deployment: Evaluating Multi-Armed Bandits in ML Ops Environments [0.0]
  We investigate whether reinforcement learning (RL)-based model management can manage deployment decisions more effectively.<n>Our approach enables more adaptive production environments by continuously evaluating deployed models and rolling back underperforming ones in real-time.<n>Our findings suggest that RL-based model management can improve automation, reduce reliance on manual interventions, and mitigate risks associated with post-deployment model failures.
  arXiv  Detail & Related papers  (2025-03-28T16:42:21Z)
• ATOM: A Framework of Detecting Query-Based Model Extraction Attacks for Graph Neural Networks [18.488168353080464]
  Graph Neural Networks (GNNs) have gained traction in Graph-based Machine Learning as a Service (GML) platforms, yet they remain vulnerable to graph-based model extraction attacks (MEAs)<n>We propose ATOM, a novel real-time MEA detection framework tailored for GNNs.<n>ATOM integrates sequential modeling and reinforcement learning to dynamically detect evolving attack patterns, while leveraging $k$core embedding to capture the structural properties, enhancing detection precision.
  arXiv  Detail & Related papers  (2025-03-20T20:25:32Z)
• AutoAct: Automatic Agent Learning from Scratch for QA via Self-Planning [54.47116888545878]
  AutoAct is an automatic agent learning framework for QA. It does not rely on large-scale annotated data and synthetic planning trajectories from closed-source models.
  arXiv  Detail & Related papers  (2024-01-10T16:57:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.