Visual-OPSD: Cross-Modal On-Policy Self-Distillation for Efficient Unified Multimodal Reasoning

• URL: http://arxiv.org/abs/2606.18974v2
• Date: Thu, 25 Jun 2026 02:14:41 GMT
• Title: Visual-OPSD: Cross-Modal On-Policy Self-Distillation for Efficient Unified Multimodal Reasoning
• Abstract summary: Unified multimodal models (UMMs) interleave generated ''visual thoughts'' (VTs) with text reasoning to improve spatial tasks.<n>This incurs roughly an order-of-magnitude inference cost from multi-step diffusion.<n>We propose Visual On-Policy Self-Distillation(Visual-OPSD) to solve this problem.
• Score: 30.851590309402436
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Unified multimodal models (UMMs) interleave generated ''visual thoughts'' (VTs) with text reasoning to improve spatial tasks. This incurs roughly an order-of-magnitude inference cost from multi-step diffusion. We find this cost yields limited direct benefit. On ThinkMorph, removing or noising VTs barely changes accuracy across nine benchmarks. Once rendered, attention concentrates on the VT regardless of content. Yet a KL diagnostic shows that conditioning on a privileged VT trace shifts the model's completion distribution. This suggests the generation pathway encodes useful reasoning beyond the rendered pixels. Motivated by this gap, we propose Visual On-Policy Self-Distillation(Visual-OPSD). Teacher and student share identical weights but differ in context: the teacher sees privileged VTs while the student sees only the question. Token-level JSD distillation on on-policy student trajectories transfers the teacher's reasoning to a text-only student. Across nine benchmarks, Visual-OPSD improves over its generative teacher by $+3.40$pp with $14.3\times$ speedup (10.0s vs. 142.8s per sample) and outperforms same-scale VLMs by $+63.83$pp on VSP. A Gaussian-noise control ($+0.40$pp vs. $+10.28$pp for real VTs) and $58.4\%$ closure of the KL gap confirm that gains come from the semantic content of the generation pathway.

Related papers

• Temporal Self-Distillation: Learning Visual State Tracking in Videos Without Supervision [22.058460912910125]
  S$3$T (Self-Supervised Self-Distillation over Time) is the first fully self-contained framework for continuous video state tracking.<n>On LLaVA-OneVision-2-8B, S$3$T improves VSTAT accuracy by $+1.74$ as a single model, $+2.38$ with souping, and $+2.70$ with additional vision-encoder adaptation.<n>The capability learned from unlabeled synthetic clips transfers to real videos, improving performance by $+7.95$ on VSTAT-YouTube state-tracking questions and $+4.50
  arXiv  Detail & Related papers  (2026-09-03T17:59:55Z)
• StreamOPD: A Post-Training Recipe with Spatio-Temporal Cue Gating for Streaming Video Understanding [16.629848186775718]
  Reinforcement learning with verifiable rewards fits this regime poorly, encouraging long think-then-answer'' generations.<n>On-policy distillation supplies dense token-level teacher supervision on student trajectories but is stable only when both models train in thinking mode.<n>We provide a transparent and reproducible reference for open-source streaming-video research.
  arXiv  Detail & Related papers  (2026-08-17T09:25:36Z)
• Visual Contrastive Self-Distillation [61.298159957622886]
  We propose Visual Contrastive Self-Distillation, which converts image-content removal into an on-policy self-distillation signal.<n>We use this contrast to sharpen the teacher's original-image distribution within its plausible support, and distill the resulting full-distribution target into the student.
  arXiv  Detail & Related papers  (2026-07-23T17:37:37Z)
• From Patches to Trajectories: Privileged Process Supervision for Software-Engineering Agents [56.31499185764872]
  Supervised fine-tuning (SFT) on long teacher trajectories is the dominant way to instill investigation and reasoning in open software-engineering (SWE) agents.<n>We propose Patches-to-Trajectories (P2T), which uses $pstar$ as privileged information during curation and formulates trajectory construction as bi-objective optimization over per-step effectiveness and trajectory length.
  arXiv  Detail & Related papers  (2026-05-21T04:54:55Z)
• HEED: Density-Weighted Residual Alignment for Hybrid Vision-Language Model Distillation [5.501291336853232]
  We distill Qwen3-VL-8B-Instruct into a 3:1 Mamba-2/attention hybrid.<n>Student model stays within 2 points of the teacher across visual reasoning benchmarks like MMStar, MMBench, and MMMU-Pro.<n>The student can still understand the scene but loses the fine-grained text needed to answer.<n>After standard post-training, the student reaches teacher-level performance on the 10-benchmark average with a 4.12$times$ throughput and a 68% memory saving at 128k context.
  arXiv  Detail & Related papers  (2026-05-16T17:33:24Z)
• Offline Semantic Guidance for Efficient Vision-Language-Action Policy Distillation [1.5718245345210466]
  We introduce textbfVLA-AD, a distillation framework that uses a Vision-Language Model as an offline semantic supervisor.<n>Instead of relying on low-level action imitation, VLA-AD augments teacher-provided 7-DoF action targets with high-level semantic guidance.
  arXiv  Detail & Related papers  (2026-05-15T17:48:25Z)
• DeltaPrompts: Escaping the Zero-Delta Trap in Multimodal Distillation [49.98710755440242]
  Distillation enables compact Vision-Language Models (VLMs) to obtain strong reasoning capabilities.<n>We reveal a critical inefficiency in this approach: up to 69% of the prompts in standard chart / document reasoning datasets are effectively zero-delta.<n>We propose a staged synthesis pipeline that repurposes existing datasets as seeds, actively targeting student failure modes to produce better prompts.
  arXiv  Detail & Related papers  (2026-05-15T02:04:12Z)
• PIO-FVLM: Rethinking Training-Free Visual Token Reduction for VLM Acceleration from an Inference-Objective Perspective [59.24570811503256]
  We propose PIO-FVLM to reduce redundant visual tokens in vision-models (VLMs) to accelerate inference.<n>The proposed PIO-FVLM is training-free and compatible with FlashAttention, friendly to practical application and deployment.<n>On LLaVA-Next-7B, PIO-FVLM retains just 11.1% of visual tokens but maintains 97.2% of the original performance.
  arXiv  Detail & Related papers  (2026-02-04T15:33:10Z)
• OVD: On-policy Verbal Distillation [47.727229201069555]
  On-policy Verbal Distillation (OVD) is a memory-efficient framework that replaces token-level probability matching with trajectory matching.<n>OVD dramatically reduces memory consumption while enabling on-policy distillation from teacher models with verbal feedback.
  arXiv  Detail & Related papers  (2026-01-29T16:48:14Z)
• VOLD: Reasoning Transfer from LLMs to Vision-Language Models via On-Policy Distillation [67.98620973023709]
  VOLD is a framework to transfer reasoning capabilities from text-only teacher models to VLM student models.<n>We show that VOLD outperforms the baseline model significantly and improves over the state of the art by a margin.
  arXiv  Detail & Related papers  (2025-10-27T16:32:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.