FiMMIA: scaling semantic perturbation-based membership inference across modalities

• URL: http://arxiv.org/abs/2512.02786v1
• Date: Tue, 02 Dec 2025 14:00:28 GMT
• Title: FiMMIA: scaling semantic perturbation-based membership inference across modalities
• Abstract summary: Membership Inference Attacks (MIAs) aim to determine whether a specific data point was included in the training set of a target model.<n>Our approach trains a neural network to analyze the target model's behavior on perturbed inputs, capturing distributional differences between members and non-members.
• Score: 1.1076204470866262
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Membership Inference Attacks (MIAs) aim to determine whether a specific data point was included in the training set of a target model. Although there are have been numerous methods developed for detecting data contamination in large language models (LLMs), their performance on multimodal LLMs (MLLMs) falls short due to the instabilities introduced through multimodal component adaptation and possible distribution shifts across multiple inputs. In this work, we investigate multimodal membership inference and address two issues: first, by identifying distribution shifts in the existing datasets, and second, by releasing an extended baseline pipeline to detect them. We also generalize the perturbation-based membership inference methods to MLLMs and release \textbf{FiMMIA} -- a modular \textbf{F}ramework for \textbf{M}ultimodal \textbf{MIA}.\footnote{The source code and framework have been made publicly available under the MIT license via \href{https://github.com/ai-forever/data_leakage_detect}{link}.The video demonstration is available on \href{https://youtu.be/a9L4-H80aSg}{YouTube}.} Our approach trains a neural network to analyze the target model's behavior on perturbed inputs, capturing distributional differences between members and non-members. Comprehensive evaluations on various fine-tuned multimodal models demonstrate the effectiveness of our perturbation-based membership inference attacks in multimodal domains.

Related papers

• A Mutual Information Lower Bound for Multimodal Regression Active Learning [9.04041860173466]
  Mutual Information Lower Bound (MI-LB) is a closed-form approximation for Mixture Network ensembles.<n>We show that MI-LB matches or beats every baseline evaluated on benchmarks featuring multimodal systems.
  arXiv  Detail & Related papers  (2026-05-14T14:50:47Z)
• G-Drift MIA: Membership Inference via Gradient-Induced Feature Drift in LLMs [1.8986796884429726]
  Membership inference attacks (MIAs) aim to determine whether a given example was used during training.<n>We introduce G-Drift MIA, a white-box membership inference method based on gradient-induced feature drift.<n>G-Drift substantially outperforms confidence-based, perplexity-based, and reference-based attacks.
  arXiv  Detail & Related papers  (2026-04-01T03:01:27Z)
• From Sparse Decisions to Dense Reasoning: A Multi-attribute Trajectory Paradigm for Multimodal Moderation [59.27094165576015]
  We propose a novel learning paradigm (UniMod) that transitions from sparse decision-making to dense reasoning traces.<n>By constructing structured trajectories encompassing evidence grounding, modality assessment, risk mapping, policy decision, and response generation, we reformulate monolithic decision tasks into a multi-dimensional boundary learning process.<n>We introduce specialized optimization strategies to decouple task-specific parameters and rebalance training dynamics, effectively resolving interference between diverse objectives in multi-task learning.
  arXiv  Detail & Related papers  (2026-01-28T09:29:40Z)
• Lost in Modality: Evaluating the Effectiveness of Text-Based Membership Inference Attacks on Large Multimodal Models [3.9448289587779404]
  Logit-based membership inference attacks (MIAs) have become a widely adopted approach for assessing data exposure in large language models (LLMs)<n>We present the first comprehensive evaluation of extending these text-based MIA methods to multimodal settings.
  arXiv  Detail & Related papers  (2025-12-02T14:11:51Z)
• Quantization Meets dLLMs: A Systematic Study of Post-training Quantization for Diffusion LLMs [78.09559830840595]
  We present the first systematic study on quantizing diffusion-based language models.<n>We identify the presence of activation outliers, characterized by abnormally large activation values.<n>We implement state-of-the-art PTQ methods and conduct a comprehensive evaluation.
  arXiv  Detail & Related papers  (2025-08-20T17:59:51Z)
• Robust Multi-View Learning via Representation Fusion of Sample-Level Attention and Alignment of Simulated Perturbation [61.64052577026623]
  Real-world multi-view datasets are often heterogeneous and imperfect.<n>We propose a novel robust MVL method (namely RML) with simultaneous representation fusion and alignment.<n>Our RML is self-supervised and can also be applied for downstream tasks as a regularization.
  arXiv  Detail & Related papers  (2025-03-06T07:01:08Z)
• Detecting Training Data of Large Language Models via Expectation Maximization [62.28028046993391]
  We introduce EM-MIA, a novel membership inference method that iteratively refines membership scores and prefix scores via an expectation-maximization algorithm.<n> EM-MIA achieves state-of-the-art results on WikiMIA.
  arXiv  Detail & Related papers  (2024-10-10T03:31:16Z)
• FedMAC: Tackling Partial-Modality Missing in Federated Learning with Cross-Modal Aggregation and Contrastive Regularization [18.276480518649404]
  Federated Learning (FL) is a method for training machine learning models using distributed data sources.<n>This study proposes a novel framework named FedMAC, designed to address multi-modality missing under conditions of partial-modality missing in FL.
  arXiv  Detail & Related papers  (2024-10-04T01:24:02Z)
• MOSAIC: Multiple Observers Spotting AI Content [35.67613230687864]
  Large Language Models (LLMs) are trained at scale and endowed with powerful text-generating abilities.<n>We propose an approach to automatically discriminate artificially generated from human-written texts.<n>Our experiments, conducted with various generator LLMs, indicate that this approach effectively leverages the strengths of each model.
  arXiv  Detail & Related papers  (2024-09-11T20:55:12Z)
• Amortized Bayesian Multilevel Models [9.831471158899644]
  Multilevel models (MLMs) are a central building block of the Bayesian workflow.<n>MLMs pose significant computational challenges, often rendering their estimation and evaluation intractable within reasonable time constraints.<n>Recent advances in simulation-based inference offer promising solutions for addressing complex probabilistic models using deep generative networks.<n>We explore a family of neural network architectures that leverage the probabilistic factorization of multilevel models to facilitate efficient neural network training and subsequent near-instant posterior inference on unseen datasets.
  arXiv  Detail & Related papers  (2024-08-23T17:11:04Z)
• Do Membership Inference Attacks Work on Large Language Models? [141.2019867466968]
  Membership inference attacks (MIAs) attempt to predict whether a particular datapoint is a member of a target model's training data. We perform a large-scale evaluation of MIAs over a suite of language models trained on the Pile, ranging from 160M to 12B parameters. We find that MIAs barely outperform random guessing for most settings across varying LLM sizes and domains.
  arXiv  Detail & Related papers  (2024-02-12T17:52:05Z)
• Practical Membership Inference Attacks against Fine-tuned Large Language Models via Self-prompt Calibration [32.15773300068426]
  Membership Inference Attacks aim to infer whether a target data record has been utilized for model training. We propose a Membership Inference Attack based on Self-calibrated Probabilistic Variation (SPV-MIA)
  arXiv  Detail & Related papers  (2023-11-10T13:55:05Z)
• Amortizing intractable inference in large language models [56.92471123778389]
  We use amortized Bayesian inference to sample from intractable posterior distributions. We empirically demonstrate that this distribution-matching paradigm of LLM fine-tuning can serve as an effective alternative to maximum-likelihood training. As an important application, we interpret chain-of-thought reasoning as a latent variable modeling problem.
  arXiv  Detail & Related papers  (2023-10-06T16:36:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.