Toward Fine-Grained Forgetting:Attribute Unlearning for Multimodal Large Language Models

• URL: http://arxiv.org/abs/2608.01008v1
• Date: Sun, 02 Aug 2026 05:23:44 GMT
• Title: Toward Fine-Grained Forgetting:Attribute Unlearning for Multimodal Large Language Models
• Abstract summary: Machine unlearning seeks to remove designated knowledge without retraining from scratch while preserving general utility.<n>Existing privacy-oriented benchmarks primarily adopt profile-level deletion, whereas practical requests are often finer grained.<n>We introduce attribute-level MLLM unlearning as a finer-grained task and construct a benchmark spanning long-text, numeric, and short-text targets, multiple forget ratios, and diverse question types.<n>Our evaluation reveals that target and retained attributes share identity-specific and visual evidence, making selective forgetting susceptible to residual leakage or collateral degradation.
• Score: 24.764929711765944
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multimodal large language models (MLLMs) exhibit strong vision--language capabilities but may also memorize and disclose sensitive information. Machine unlearning seeks to remove designated knowledge without retraining from scratch while preserving general utility. Existing privacy-oriented benchmarks primarily adopt profile-level deletion, whereas practical requests are often finer grained: a model should forget a specified attribute while retaining non-sensitive information about the same identity. We therefore introduce attribute-level MLLM unlearning as a finer-grained task and construct a benchmark spanning long-text, numeric, and short-text targets, multiple forget ratios, and diverse question types. Our evaluation reveals that target and retained attributes share identity-specific and visual evidence, making selective forgetting susceptible to residual leakage or collateral degradation; accordingly, existing methods exhibit unstable forgetting--retention trade-offs in this setting. To address this challenge, we propose Causal Localization and Retain-Aware Projection (CLRP), a lightweight training-free framework. CLRP uses activation patching to identify the layer that causally mediates target-attribute disclosure, then applies a retain-aware projection that removes the target-attribute subspace while preserving same-identity evidence. Experiments across multiple widely used MLLMs with distinct architectures and parameter scales demonstrate the effectiveness of CLRP.

Related papers

• Where Identity Lives: Localized, Retain-Free Identity Unlearning in Multimodal Large Language Models [33.31500669671185]
  Forgetting from the forget set alone damages the shared visual-language computation, harming perception.<n>We turn this into Pathway-Aware Visual-attribute Anchoring (PAVA), which updates to these layers and pairs a forget loss with a visual-attribute anchor that preserves image-grounded behavior by distilling the model's own pre-unlearning answers from the forget images alone.<n>On MLLMU-Bench and ReMem, PAVA gives the strongest forget-retain trade-off among forget-set-only methods and remains competitive with retain-based baselines.
  arXiv  Detail & Related papers  (2026-08-31T11:51:05Z)
• Memory-Augmented LLM-based Multi-Agent System for Automated Feature Generation on Tabular Data [52.80567461420316]
  We propose a Memory-Augmented LLM-based Multi-Agent System (textbfMALMAS) for automated feature generation.<n>MALMAS decomposes the generation process into agents with distinct responsibilities, and a Router Agent activates an appropriate subset of agents per iteration.<n>We further integrate a memory module comprising procedural memory, feedback memory, and conceptual memory, enabling iterative refinement that adaptively guides subsequent feature generation.
  arXiv  Detail & Related papers  (2026-04-22T07:09:30Z)
• MeGU: Machine-Guided Unlearning with Target Feature Disentanglement [73.49657372882082]
  We propose a novel framework that guides unlearning through concept-aware re-alignment.<n>MeGU enables controlled and selective forgetting, effectively mitigating both under-unlearning and over-unlearning.
  arXiv  Detail & Related papers  (2026-02-19T05:20:31Z)
• What Makes You Unique? Attribute Prompt Composition for Object Re-Identification [70.67907354506278]
  Object Re-IDentification aims to recognize individuals across non-overlapping camera views.<n>Single-domain models tend to overfit to domain-specific features, whereas cross-domain models often rely on diverse normalization strategies.<n>We propose an Attribute Prompt Composition framework, which exploits textual semantics to jointly enhance discrimination and generalization.
  arXiv  Detail & Related papers  (2025-09-23T07:03:08Z)
• GUARD: Generation-time LLM Unlearning via Adaptive Restriction and Detection [36.38245533018162]
  Large Language Models (LLMs) have demonstrated strong capabilities in memorizing vast amounts of knowledge across diverse domains.<n>Existing unlearning efforts typically fine-tune the model with resources such as forget data, retain data, and a calibration model.<n>We propose Generation-time Unlearning via Adaptive Restriction and Detection (GUARD), a framework that enables dynamic unlearning during LLM generation.
  arXiv  Detail & Related papers  (2025-05-19T16:26:58Z)
• Unlearning Sensitive Information in Multimodal LLMs: Benchmark and Attack-Defense Evaluation [88.78166077081912]
  We introduce a multimodal unlearning benchmark, UnLOK-VQA, and an attack-and-defense framework to evaluate methods for deleting specific multimodal knowledge from MLLMs.<n>Our results show multimodal attacks outperform text- or image-only ones, and that the most effective defense removes answer information from internal model states.
  arXiv  Detail & Related papers  (2025-05-01T01:54:00Z)
• Self-Regularization with Sparse Autoencoders for Controllable LLM-based Classification [29.74457390987092]
  We propose a novel framework to identify and regularize unintended features in large language models (LLMs) latent spaces.<n>We evaluate the proposed framework on three real-world tasks, including toxic chat detection, reward modeling, and disease diagnosis.
  arXiv  Detail & Related papers  (2025-02-19T22:27:59Z)
• Efficiently Disentangling CLIP for Multi-Object Perception [62.523137132812764]
  Vision-language models like CLIP excel at recognizing the single, prominent object in a scene, but struggle in complex scenes containing multiple objects.<n>We propose DCLIP, an efficient framework that learns an optimal level of mutual information while adding only minimal learnable parameters to a frozen VLM.
  arXiv  Detail & Related papers  (2025-02-05T08:20:31Z)
• RAR: Retrieving And Ranking Augmented MLLMs for Visual Recognition [78.97487780589574]
  Multimodal Large Language Models (MLLMs) excel at classifying fine-grained categories. This paper introduces a Retrieving And Ranking augmented method for MLLMs. Our proposed approach not only addresses the inherent limitations in fine-grained recognition but also preserves the model's comprehensive knowledge base.
  arXiv  Detail & Related papers  (2024-03-20T17:59:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.