AIM: Anchor Identity Features, Then Match for Multimodal Large Language Model Unlearning

• URL: http://arxiv.org/abs/2608.28312v1
• Date: Fri, 28 Aug 2026 13:22:22 GMT
• Title: AIM: Anchor Identity Features, Then Match for Multimodal Large Language Model Unlearning
• Abstract summary: We study identity unlearning when retain images are unavailable at deletion time.<n>Our analysis shows that identity and visual-perception questions occupy distinct regions in fine-tuned hidden states.<n>We propose AIM, a two-stage method that anchors an identity-forgetting target with a universal visual prompt and then matches the vision encoder to that target.
• Score: 33.31500669671185
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multimodal large language models (MLLMs) can memorize identity-specific facts about people in their fine-tuning data, creating privacy risks when a person requests deletion. Existing MLLM unlearning methods often assume access to retain images or ground-truth answers during deletion, which is unrealistic in many practical scenarios. We study identity unlearning when retain images are unavailable at deletion time. Our analysis shows that identity and visual-perception questions occupy distinct regions in fine-tuned hidden states and are organized differently: identity questions cluster by person, whereas perception questions cluster by question type. This suggests that identity knowledge can be suppressed without erasing general visual perception. Building on this observation, we propose AIM, a two-stage method that anchors an identity-forgetting target with a universal visual prompt and then matches the vision encoder to that target under a Fisher-based constraint. Extensive experiments show that AIM achieves competitive identity forgetting while preserving non-deleted identities, prior knowledge, and visual perception on the same images.

Related papers

• Where Identity Lives: Localized, Retain-Free Identity Unlearning in Multimodal Large Language Models [33.31500669671185]
  Forgetting from the forget set alone damages the shared visual-language computation, harming perception.<n>We turn this into Pathway-Aware Visual-attribute Anchoring (PAVA), which updates to these layers and pairs a forget loss with a visual-attribute anchor that preserves image-grounded behavior by distilling the model's own pre-unlearning answers from the forget images alone.<n>On MLLMU-Bench and ReMem, PAVA gives the strongest forget-retain trade-off among forget-set-only methods and remains competitive with retain-based baselines.
  arXiv  Detail & Related papers  (2026-08-31T11:51:05Z)
• Toward Fine-Grained Forgetting:Attribute Unlearning for Multimodal Large Language Models [24.764929711765944]
  Machine unlearning seeks to remove designated knowledge without retraining from scratch while preserving general utility.<n>Existing privacy-oriented benchmarks primarily adopt profile-level deletion, whereas practical requests are often finer grained.<n>We introduce attribute-level MLLM unlearning as a finer-grained task and construct a benchmark spanning long-text, numeric, and short-text targets, multiple forget ratios, and diverse question types.<n>Our evaluation reveals that target and retained attributes share identity-specific and visual evidence, making selective forgetting susceptible to residual leakage or collateral degradation.
  arXiv  Detail & Related papers  (2026-08-02T05:23:44Z)
• Vera: Identity-Faithful Human Subject-to-Video Generation [66.45205582225942]
  We propose Vera, a unified human-centric S2V framework for single- and multi-person generation.<n>We first construct a million-pair identity-aligned human image-video dataset through person-level cross-clip retrieval.<n>We show that Vera improves human identity consistency, multi-person subject binding, and motion naturalness, while reducing identity confusion and excessive reference-image copying.
  arXiv  Detail & Related papers  (2026-07-22T15:08:30Z)
• PIU: Proximity-guided Identity Unlearning in ID-Conditioned Diffusion Models [2.260041671242786]
  We study identity unlearning in Arc2Face, a state-of-the-art identity-conditioned latent diffusion model for face generation.<n>We introduce Proximity-guided Identity Unlearning (PIU), an anchor-guided framework for identity unlearning.
  arXiv  Detail & Related papers  (2026-05-21T10:55:56Z)
• Null Space Constrained Contrastive Visual Forgetting for MLLM Unlearning [22.667378623339527]
  We introduce an MLLM unlearning approach that aims to forget target visual knowledge while preserving non-target visual knowledge and all textual knowledge.<n>First, we propose a Contrastive Visual Forgetting (CVF) mechanism to separate target visual knowledge from retained visual knowledge.<n>Second, we identify the null space associated with retained knowledge and constrain the unlearning process within this space, thereby significantly mitigating degradation in knowledge retention.
  arXiv  Detail & Related papers  (2026-05-07T09:18:58Z)
• ViCrit: A Verifiable Reinforcement Learning Proxy Task for Visual Perception in VLMs [98.27348724529257]
  We introduce ViCrit (Visual Caption Hallucination Critic), an RL proxy task that trains VLMs to localize a subtle, synthetic visual hallucination injected into paragraphs of human-written image captions.<n>Models trained with the ViCrit Task exhibit substantial gains across a variety of vision-language models benchmarks.
  arXiv  Detail & Related papers  (2025-06-11T19:16:54Z)
• IDA-VLM: Towards Movie Understanding via ID-Aware Large Vision-Language Model [52.697180472760635]
  This paper explores the potential of character identities memory and recognition across multiple visual scenarios. We propose visual instruction tuning with ID reference and develop an ID-Aware Large Vision-Language Model, IDA-VLM. Our research introduces a novel benchmark MM-ID, to examine LVLMs on instance IDs memory and recognition across four dimensions.
  arXiv  Detail & Related papers  (2024-07-10T12:11:59Z)
• Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs [50.77984109941538]
  Our research reveals that the visual capabilities in recent multimodal LLMs still exhibit systematic shortcomings. We identify ''CLIP-blind pairs'' - images that CLIP perceives as similar despite their clear visual differences. We evaluate various CLIP-based vision-and-language models and found a notable correlation between visual patterns that challenge CLIP models and those problematic for multimodal LLMs.
  arXiv  Detail & Related papers  (2024-01-11T18:58:36Z)
• Disentangle Before Anonymize: A Two-stage Framework for Attribute-preserved and Occlusion-robust De-identification [55.741525129613535]
  "Disentangle Before Anonymize" is a novel two-stage Framework(DBAF)<n>This framework includes a Contrastive Identity Disentanglement (CID) module and a Key-authorized Reversible Identity Anonymization (KRIA) module.<n>Extensive experiments demonstrate that our method outperforms state-of-the-art de-identification approaches.
  arXiv  Detail & Related papers  (2023-11-15T08:59:02Z)
• Diff-Privacy: Diffusion-based Face Privacy Protection [58.1021066224765]
  In this paper, we propose a novel face privacy protection method based on diffusion models, dubbed Diff-Privacy. Specifically, we train our proposed multi-scale image inversion module (MSI) to obtain a set of SDM format conditional embeddings of the original image. Based on the conditional embeddings, we design corresponding embedding scheduling strategies and construct different energy functions during the denoising process to achieve anonymization and visual identity information hiding.
  arXiv  Detail & Related papers  (2023-09-11T09:26:07Z)
• Uncertainty-Aware Multi-Shot Knowledge Distillation for Image-Based Object Re-Identification [93.39253443415392]
  We propose exploiting the multi-shots of the same identity to guide the feature learning of each individual image. It consists of a teacher network (T-net) that learns the comprehensive features from multiple images of the same object, and a student network (S-net) that takes a single image as input. We validate the effectiveness of our approach on the popular vehicle re-id and person re-id datasets.
  arXiv  Detail & Related papers  (2020-01-15T09:39:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.