Exponential-Family Membership Inference: From LiRA and RMIA to BaVarIA

• URL: http://arxiv.org/abs/2603.11799v1
• Date: Thu, 12 Mar 2026 11:00:19 GMT
• Title: Exponential-Family Membership Inference: From LiRA and RMIA to BaVarIA
• Abstract summary: Membership inference attacks (MIAs) are becoming standard tools for auditing the privacy of machine learning models.<n>We show that all three are instances of a single exponential-family log-likelihood ratio framework, differing only in their distributional assumptions.<n>We propose BaVarIA, a Bayesian variance inference attack that replaces threshold-based parameter switching with conjugate normal-inverse-gamma priors.
• Score: 4.56877715768796
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Membership inference attacks (MIAs) are becoming standard tools for auditing the privacy of machine learning models. The leading attacks -- LiRA (Carlini et al., 2022) and RMIA (Zarifzadeh et al., 2024) -- appear to use distinct scoring strategies, while the recently proposed BASE (Lassila et al., 2025) was shown to be equivalent to RMIA, making it difficult for practitioners to choose among them. We show that all three are instances of a single exponential-family log-likelihood ratio framework, differing only in their distributional assumptions and the number of parameters estimated per data point. This unification reveals a hierarchy (BASE1-4) that connects RMIA and LiRA as endpoints of a spectrum of increasing model complexity. Within this framework, we identify variance estimation as the key bottleneck at small shadow-model budgets and propose BaVarIA, a Bayesian variance inference attack that replaces threshold-based parameter switching with conjugate normal-inverse-gamma priors. BaVarIA yields a Student-t predictive (BaVarIA-t) or a Gaussian with stabilized variance (BaVarIA-n), providing stable performance without additional hyperparameter tuning. Across 12 datasets and 7 shadow-model budgets, BaVarIA matches or improves upon LiRA and RMIA, with the largest gains in the practically important low-shadow-model and offline regimes.

Related papers

• BaRA: Bayesian Adaptive Rank Allocation for Parameter-Efficient Fine-Tuning [40.998095775740886]
  Low-rank adaptation (LoRA) enables highly efficient fine-tuning by constraining task-specific updates to fixed low-rank subspaces.<n>We propose BaRA, a Bayesian Adaptive Rank Allocation framework for parameter-efficient fine-tuning.<n>BaRA dynamically allocates adaptation capacity by activating a sparse, context-dependent subset of disentangled latent factors.
  arXiv  Detail & Related papers  (2026-06-28T04:08:09Z)
• Joint Model and Data Sparsification via the Marginal Likelihood [53.29070892356214]
  We propose jointly learning individual feature and sample relevancies, enabling simultaneous model and data sparsification.<n>This symmetric pruning of model and data offers a natural extension that preserves conjugacy.<n> Empirical results across diverse regression tasks affirm that a joint ARD approach consistently yields both sparse and robust prediction models.
  arXiv  Detail & Related papers  (2026-05-28T13:26:53Z)
• Bayesian Model Merging [17.887004278413915]
  Model merging aims to combine multiple task-specific expert models into a single model without joint retraining.<n>This paper introduces Bayesian Model Merging (BMM), a plug-and-play bi-level optimization framework.<n>BMM consistently outperforms all plug-and-play anchor baselines.
  arXiv  Detail & Related papers  (2026-05-13T00:36:47Z)
• IRIS: Interpolative Rényi Iterative Self-play for Large Language Model Fine-Tuning [1.4474373238664187]
  Self-play fine-tuning enables large language models to improve beyond supervised fine-tuning without additional human annotations.<n>We propose IRIS (Interpolative Rényi Iterative Self-play), a Rényi-based self-play fine-tuning framework with a continuously adjustable objective.<n>Experiments on Zephyr-7B and Qwen2.5-3B across ten benchmarks show that IRIS improves upon baselines, reaching 44.57% average score with gains across iterations.
  arXiv  Detail & Related papers  (2026-04-22T11:52:21Z)
• Too Correct to Learn: Reinforcement Learning on Saturated Reasoning Data [55.84428098924793]
  We propose Constrained Uniform Top-K Sampling (CUTS), a parameter-free decoding strategy enforcing structure-preserving exploration.<n>We integrate this into Mixed-CUTS, a training framework synergizing exploitative and exploratory rollouts to amplify intra-group advantage variance.<n> Notably, Mixed-CUTS improves Pass@1 accuracy on the challenging AIME25 benchmark by up to 15.1% over standard GRPO.
  arXiv  Detail & Related papers  (2026-04-20T16:43:28Z)
• $V_{0.5}$: Generalist Value Model as a Prior for Sparse RL Rollouts [81.48669089692189]
  Generalist Value Models (such as $V_0.5$) achieve pre-trained value estimation by explicitly encoding model capabilities in-context.<n>We propose $V_0.5$, which adaptively fuses the baseline predicted by such value model with the empirical mean derived from sparse rollouts.<n>$V_0.5$ significantly outperforms GRPO and DAPO, achieving faster convergence and over some 10% performance improvement.
  arXiv  Detail & Related papers  (2026-03-11T14:57:41Z)
• Adjustment of Cluster-Then-Predict Framework for Multiport Scatterer Load Prediction [41.99844472131922]
  Predicting interdependent load values in multiport scatterers is challenging due to high dimensionality and complex dependence between impedance and scattering ability.<n>We propose a two-stage cluster-then-predict framework for multiple load values prediction task in multiport scatterers.
  arXiv  Detail & Related papers  (2026-02-08T21:26:05Z)
• Robust Variational Bayes by Min-Max Median Aggregation [13.102667562202386]
  We propose a robust variational Bayes framework to handle contamination and outliers in dataset.<n>Our approach partitions the data into $m$ disjoint subsets and formulates a joint optimization problem based on robust aggregation principles.<n>Our findings indicate that the two-stage approach yields a smaller approximation error compared to directly aggregating the $m$-powered local posteriors.
  arXiv  Detail & Related papers  (2025-12-14T13:02:00Z)
• MMR1: Enhancing Multimodal Reasoning with Variance-Aware Sampling and Open Resources [113.33902847941941]
  Variance-Aware Sampling (VAS) is a data selection strategy guided by Variance Promotion Score (VPS)<n>We release large-scale, carefully curated resources containing 1.6M long CoT cold-start data and 15k RL QA pairs.<n> Experiments across mathematical reasoning benchmarks demonstrate the effectiveness of both the curated data and the proposed VAS.
  arXiv  Detail & Related papers  (2025-09-25T14:58:29Z)
• Practical Bayes-Optimal Membership Inference Attacks [51.06337749378432]
  We develop practical and theoretically grounded membership inference attacks (MIAs) against both independent and identically distributed (i.i.d.) data and graph-structured data.<n>Building on the Bayesian decision-theoretic framework of Sablayrolles et al., we derive the Bayes-optimal membership inference rule for node-level MIAs against graph neural networks.
  arXiv  Detail & Related papers  (2025-05-30T00:23:01Z)
• VAPO: Efficient and Reliable Reinforcement Learning for Advanced Reasoning Tasks [49.0793012627959]
  We present VAPO, a novel framework tailored for reasoning models within the value-based paradigm.<n>VAPO attains a state-of-the-art score of $mathbf60.4$.<n>In direct comparison under identical experimental settings, VAPO outperforms the previously reported results of DeepSeek-R1-Zero-Qwen-32B and DAPO by more than 10 points.
  arXiv  Detail & Related papers  (2025-04-07T14:21:11Z)
• HOME: High-Order Mixed-Moment-based Embedding for Representation Learning [6.693379403133435]
  We propose the High-Order Mixed-Moment-based Embedding (HOME) strategy to reduce redundancy between any sets of feature variables. Our initial experiments show that a simple version in the form of a three-order HOME scheme already significantly outperforms the current two-order baseline method.
  arXiv  Detail & Related papers  (2022-07-15T20:34:49Z)
• BCDAG: An R package for Bayesian structure and Causal learning of Gaussian DAGs [77.34726150561087]
  We introduce the R package for causal discovery and causal effect estimation from observational data. Our implementation scales efficiently with the number of observations and, whenever the DAGs are sufficiently sparse, the number of variables in the dataset. We then illustrate the main functions and algorithms on both real and simulated datasets.
  arXiv  Detail & Related papers  (2022-01-28T09:30:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.