Visual-Noise Guided In-Context Distillation for Multimodal Large Language Model Unlearning

• URL: http://arxiv.org/abs/2606.00105v1
• Date: Tue, 26 May 2026 12:49:34 GMT
• Title: Visual-Noise Guided In-Context Distillation for Multimodal Large Language Model Unlearning
• Abstract summary: Multimodal Large Language Models (MLLMs) have achieved remarkable progress on vision-language tasks.<n>They may also memorize and expose sensitive or restricted knowledge, raising concerns about privacy and broader safety risks.<n>We propose Visual-Noise Guided In-Context Distillation (VGID) as a distillation-based framework for MLLM unlearning.
• Score: 25.860185076316508
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multimodal Large Language Models (MLLMs) have achieved remarkable progress on vision-language tasks, but they may also memorize and expose sensitive or restricted knowledge, raising concerns about privacy and broader safety risks. Machine Unlearning (MU) provides a promising way to remove targeted undesirable knowledge from trained models without retraining from scratch while preserving general model utility. Nevertheless, effective unlearning in MLLMs remains particularly challenging. Existing training-based methods often struggle to balance unlearning effectiveness and model utility. In contrast, training-free methods such as in-context unlearning preserve model utility by avoiding parameter updates, but they do not remove memorized knowledge at the parameter level and may remain vulnerable to reverse-engineering attacks. More importantly, in-context unlearning is insufficient in multimodal settings, where visual inputs can provide strong conditioning signals and induce undesirable outputs. To address these challenges, we propose Visual-Noise Guided In-Context Distillation (VGID), a distillation-based framework for MLLM unlearning. VGID dynamically constructs an unlearning-oriented teacher distribution from the frozen base model through dual-modal intervention that combines visual perturbation with textual in-context unlearning. The resulting intervention-induced distribution serves as a teacher signal for distillation, guiding the student model toward parameter-level unlearning without requiring external teacher models or explicit undesirable response annotations. Experimental results show that VGID achieves strong unlearning effectiveness while preserving competitive model utility, reducing forget set ROUGE-L by 0.371 with only a 0.055 drop in retain set ROUGE-L in a representative setting.

Related papers

• ASRU: Activation Steering Meets Reinforcement Unlearning for Multimodal Large Language Models [16.973117214946107]
  Multimodal large language models (MLLMs) may memorize sensitive cross-modal information during pretraining.<n>We propose ASRU, a controllable multimodal unlearning framework that incorporates generation quality as a core evaluation objective.
  arXiv  Detail & Related papers  (2026-05-15T07:22:43Z)
• Harmonizing Multi-Objective LLM Unlearning via Unified Domain Representation and Bidirectional Logit Distillation [11.723777730116831]
  Large Language Models (LLMs) unlearning is crucial for removing hazardous or privacy-leaking information from the model.<n>Existing unlearning methods primarily focus on a limited subset of these goals.<n>We propose a novel multi-objective unlearning framework that harmonizes multiple unlearning objectives.
  arXiv  Detail & Related papers  (2026-04-16T19:09:17Z)
• Sparsity-Aware Unlearning for Large Language Models [20.699929903336113]
  Large Language Models (LLMs) inevitably memorize sensitive information during training, posing significant privacy risks.<n>Machine unlearning has emerged as a promising solution to selectively remove such information without full retraining.<n>We find that unlearning effectiveness degrades substantially on sparse models.<n>We propose Sparsity-Aware Unlearning (SAU), which decouples unlearning from sparsification objectives through gradient masking.
  arXiv  Detail & Related papers  (2026-01-31T07:45:30Z)
• DUET: Distilled LLM Unlearning from an Efficiently Contextualized Teacher [5.406594712642111]
  Distilled Unlearning from an Efficient Teacher (DUET) is a novel distillation-based unlearning method.<n>It achieves higher performance in both forgetting and utility preservation, while being orders of magnitude more data-efficient than state-of-the-art unlearning methods.
  arXiv  Detail & Related papers  (2026-01-29T05:32:35Z)
• AUVIC: Adversarial Unlearning of Visual Concepts for Multi-modal Large Language Models [63.05306474002547]
  Regulatory frameworks mandating the 'right to be forgotten' drive the need for machine unlearning.<n>We introduce AUVIC, a novel visual concept unlearning framework for MLLMs.<n>We show that AUVIC achieves state-of-the-art target forgetting rates while incurs minimal performance degradation on non-target concepts.
  arXiv  Detail & Related papers  (2025-11-14T13:35:32Z)
• MLLMEraser: Achieving Test-Time Unlearning in Multimodal Large Language Models through Activation Steering [36.80441487363007]
  MLLMEraser is an input-aware, training-free framework for test-time unlearning.<n>We construct a multimodal erasure direction by contrasting adversarially perturbed, knowledge-recall image-text pairs.<n>Experiments on LLaVA-1.5 and Qwen-2.5-VL demonstrate that MLLMEraser consistently outperforms state-of-the-art MLLM unlearning baselines.
  arXiv  Detail & Related papers  (2025-10-05T14:20:17Z)
• UniErase: Towards Balanced and Precise Unlearning in Language Models [69.04923022755547]
  Large language models (LLMs) require iterative updates to address the outdated information problem.<n>UniErase is a novel unlearning framework that demonstrates precision and balanced performances between knowledge unlearning and ability retaining.
  arXiv  Detail & Related papers  (2025-05-21T15:53:28Z)
• Learn while Unlearn: An Iterative Unlearning Framework for Generative Language Models [52.40798352740857]
  We introduce the Iterative Contrastive Unlearning (ICU) framework, which consists of three core components.<n>A Knowledge Unlearning Induction module targets specific knowledge for removal using an unlearning loss.<n>A Contrastive Learning Enhancement module preserves the model's expressive capabilities against the pure unlearning goal.<n>An Iterative Unlearning Refinement module dynamically adjusts the unlearning process through ongoing evaluation and updates.
  arXiv  Detail & Related papers  (2024-07-25T07:09:35Z)
• R-SFLLM: Jamming Resilient Framework for Split Federated Learning with Large Language Models [65.04475956174959]
  Split federated learning (SFL) is a compute-efficient paradigm in distributed machine learning (ML)<n>A significant challenge in SFL, particularly when deployed over wireless channels, is the susceptibility of transmitted model parameters to adversarial jamming.<n>This paper develops a physical layer framework for resilient SFL with large language models (LLMs) and vision language models (VLMs) over wireless networks.
  arXiv  Detail & Related papers  (2024-07-16T12:21:29Z)
• Offset Unlearning for Large Language Models [49.851093293780615]
  delta-Unlearning is an offset unlearning framework for black-box LLMs.<n>We show that delta-Unlearning can effectively unlearn target data while maintaining similar or even stronger performance on general out-of-forget-scope tasks.
  arXiv  Detail & Related papers  (2024-04-17T03:39:51Z)
• Rethinking Machine Unlearning for Large Language Models [85.92660644100582]
  We explore machine unlearning in the domain of large language models (LLMs)<n>This initiative aims to eliminate undesirable data influence (e.g., sensitive or illegal information) and the associated model capabilities.
  arXiv  Detail & Related papers  (2024-02-13T20:51:58Z)
• Unlearn What You Want to Forget: Efficient Unlearning for LLMs [92.51670143929056]
  Large language models (LLMs) have achieved significant progress from pre-training on and memorizing a wide range of textual data. This process might suffer from privacy issues and violations of data protection regulations. We propose an efficient unlearning framework that could efficiently update LLMs without having to retrain the whole model after data removals.
  arXiv  Detail & Related papers  (2023-10-31T03:35:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.