Perception Before Supervision: Self-Contained Visual Distillation from Counterfactual Blind Spots

• URL: http://arxiv.org/abs/2608.09931v1
• Date: Mon, 10 Aug 2026 17:59:56 GMT
• Title: Perception Before Supervision: Self-Contained Visual Distillation from Counterfactual Blind Spots
• Abstract summary: textbfCVPD (Contrastive Counterfactual Visual Process Distillation) is the first fully self-contained framework for dense, on-policy, token-level visual self-distillation for MLLMs.<n>CVPD identifies visual blind spots where zooming into a region changes and sharpens the model's answer distribution, while removing the same region leaves the full-image behavior largely unchanged.
• Score: 28.035029653746335
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Self-improvement for multimodal large language models (MLLMs) is typically driven by reward-based methods that provide only coarse scalar feedback. Distillation offers a richer alternative through dense token-level supervision, but in the visual domain it usually depends on privileged context constructed using external annotations and tools, or stronger models. We introduce \textbf{CVPD} (Contrastive Counterfactual Visual Process Distillation), which, to the best of our knowledge, is the first fully self-contained framework for dense, on-policy, token-level visual self-distillation for MLLMs. CVPD identifies visual blind spots where zooming into a region changes and sharpens the model's answer distribution, while removing the same region leaves the full-image behavior largely unchanged. Such regions reveal perceptual information that the model can encode but fails to consistently utilize under full-image conditioning. We propose a three-gate Counterfactual Criterion that identifies these regions directly from the model's own responses and converts them into dense contrastive supervision for self-distillation. On Qwen3-VL-8B-Instruct, CVPD outperforms six self-evolving baselines across twelve benchmarks, including methods that rely on external GPT-4o supervision, without a single regression. It achieves gains of $+3.60$ on OCRBench, $+3.38$ on MMStar Fine-Grained Perception, and $+3.08$ on MMStar Logical Reasoning, while maintaining or improving performance on broader multimodal benchmarks.

Related papers

• Failure-Informed Image Self-Augmentation for Multimodal Large Language Model Self-Improvement [50.97806700110183]
  Failure-informed Image Self-Augmentation (textbfFISA) is a framework for MLLM self-improvement that constructs augmented images from the model's own failure cases.<n>Our method generates visually challenging yet answer-preserving image complications, verifies their utility through self-examination, and applies dual fidelity filtering to avoid semantic distortion.
  arXiv  Detail & Related papers  (2026-08-04T14:28:42Z)
• Paying More Attention to Visual Tokens in Self-Evolving Large Multimodal Models [54.097381112138]
  Current self-evolving LMMs struggle on vision-language understanding tasks such as image captioning and visual question answering.<n>We propose VISE, a purely unsupervised self-evolving framework that directly regularizes the model's visual conditioning policy.<n>VISE operates within a single model without specialist roles, external reward models, or annotations.
  arXiv  Detail & Related papers  (2026-06-25T17:59:55Z)
• DeltaRubric: Generative Multimodal Reward Modeling via Joint Planning and Verification [38.65432183778467]
  We introduce DeltaRubric, an approach that reformulates multimodal preference evaluation as a plan-and-execute process within a single MLLM.<n>DeltaRubric operates in two steps: acting first as a $textitDisagreement Planner$, it generates a neutral, instance-specific verification checklist.<n>We formulate DeltaRubric as a multi-role reinforcement learning problem, jointly optimizing planning and verification capabilities.
  arXiv  Detail & Related papers  (2026-05-10T02:32:19Z)
• Difference Feedback: Generating Multimodal Process-Level Supervision for VLM Reinforcement Learning [10.473880989969844]
  We propose Differential Feedback, which automatically constructs token/step-level supervision masks by repairing erroneous reasoning trajectories.<n>Our approach offers an effective, low-cost solution for accurate vision--reasoning process alignment.
  arXiv  Detail & Related papers  (2026-03-29T02:31:12Z)
• Learning Accurate Segmentation Purely from Self-Supervision [87.78965637247107]
  Selfment is a fully self-supervised framework that segments objects directly from raw images without human labels.<n>Selfment sets new state-of-the-art (SoTA) results across multiple benchmarks.
  arXiv  Detail & Related papers  (2026-02-27T07:36:32Z)
• Thinking with Deltas: Incentivizing Reinforcement Learning via Differential Visual Reasoning Policy [75.66913260900726]
  Reinforcement Learning with Verifiable Rewards has significantly advanced reasoning capabilities in Large Language Models.<n>Existing paradigms, driven by text-centric outcome rewards, encourage models to bypass visual perception.<n>We propose textbfThinking with Deltas, a framework driven by a textbfDifferential Visual Reasoning Policy.
  arXiv  Detail & Related papers  (2026-01-11T08:25:34Z)
• VOGUE: Guiding Exploration with Visual Uncertainty Improves Multimodal Reasoning [62.09195763860549]
  Reinforcement learning with verifiable rewards (RLVR) improves reasoning in large language models (LLMs) but struggles with exploration.<n>We introduce $textbfVOGUE (Visual Uncertainty Guided Exploration)$, a novel method that shifts exploration from the output (text) to the input (visual) space.<n>Our work shows that grounding exploration in the inherent uncertainty of visual inputs is an effective strategy for improving multimodal reasoning.
  arXiv  Detail & Related papers  (2025-10-01T20:32:08Z)
• GrAInS: Gradient-based Attribution for Inference-Time Steering of LLMs and VLMs [56.93583799109029]
  GrAInS is an inference-time steering approach that operates across both language-only and vision-language models and tasks.<n>During inference, GrAInS hidden activations at transformer layers guided by token-level attribution signals, and normalizes activations to preserve representational scale.<n>It consistently outperforms both fine-tuning and existing steering baselines.
  arXiv  Detail & Related papers  (2025-07-24T02:34:13Z)
• Dual-Stage Value-Guided Inference with Margin-Based Reward Adjustment for Fast and Faithful VLM Captioning [23.851747078717473]
  We introduce textbfValue-guided Inference with Margin-based Reward (ViMaR), a two-stage inference framework that improves both efficiency and output fidelity.<n>ViMaR generates captions that are significantly more reliable, factually accurate, detailed, and explanatory, while achieving over 4$times$ speedup.
  arXiv  Detail & Related papers  (2025-06-18T17:23:36Z)
• Backdoor Cleaning without External Guidance in MLLM Fine-tuning [76.82121084745785]
  Believe Your Eyes (BYE) is a data filtering framework that leverages attention entropy patterns as self-supervised signals to identify and filter backdoor samples.<n>It achieves near-zero attack success rates while maintaining clean-task performance.
  arXiv  Detail & Related papers  (2025-05-22T17:11:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.