Persistent Visual Memory: Sustaining Perception for Deep Generation in LVLMs

• URL: http://arxiv.org/abs/2605.00814v2
• Date: Fri, 08 May 2026 16:52:48 GMT
• Title: Persistent Visual Memory: Sustaining Perception for Deep Generation in LVLMs
• Abstract summary: Persistent Visual Memory is a lightweight learnable module designed to strengthen sustained, on-demand access to visual evidence.<n>Experiments on Qwen3-VL models demonstrate that PVM brings notable improvements with negligible parameter overhead.<n>In-depth analysis reveals that PVM shows improved robustness in longer generations and accelerates internal prediction convergence.
• Score: 66.15429821530503
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While autoregressive Large Vision-Language Models (LVLMs) demonstrate remarkable proficiency in multimodal tasks, they face a "Visual Signal Dilution" phenomenon, where the accumulation of textual history expands the attention partition function, causing visual attention to decay inversely with generated sequence length. To counteract this, we propose Persistent Visual Memory (PVM), a lightweight learnable module designed to strengthen sustained, on-demand access to visual evidence. Integrated as a parallel branch alongside the Feed-Forward Network (FFN) in LVLMs, PVM establishes a distance-agnostic retrieval pathway that directly provides visual embeddings for enhanced visual perception, thereby structurally mitigating the signal suppression inherent to deep generation. Extensive experiments on Qwen3-VL models demonstrate that PVM brings notable improvements with negligible parameter overhead, delivering consistent average accuracy gains across both 4B and 8B scales, particularly in complex reasoning tasks that demand persistent visual perception. Furthermore, in-depth analysis reveals that PVM shows improved robustness in longer generations and accelerates internal prediction convergence.

Related papers

• HAFI-VLM: A Frequency Perspective for Diagnosing and Enhancing Visual Perception in Vision-Language Models [9.926121668688772]
  Vision-language models (VLMs) remain unreliable when predictions require fine-grained visual evidence.<n>We identify a previously overlooked cause: spectral response rigidity.<n>We propose HAFI-VLM, which introduces a task-conditioned frequency pathway while preserving the pretrained semantic representation.
  arXiv  Detail & Related papers  (2026-08-03T12:15:33Z)
• From Attenuation to Attention: Variational Information Flow Manipulation for Fine-Grained Visual Perception [12.4823697158657]
  Multimodal Large Language Models (MLLMs) have demonstrated impressive capabilities in general visual understanding.<n>They frequently falter in fine-grained perception tasks that require identifying tiny objects or discerning subtle visual relationships.<n>We attribute this limitation to Visual Attenuation: a phenomenon where sparse fine-grained visual signals are prematurely suppressed or diluted by dominant textual tokens during network propagation.<n>Existing input-centric solutions fail to fundamentally reverse this intrinsic mechanism of information loss.<n>We propose the Variational Information Flow (VIF) framework to address this challenge.
  arXiv  Detail & Related papers  (2026-04-14T09:32:13Z)
• Cognitive Pivot Points and Visual Anchoring: Unveiling and Rectifying Hallucinations in Multimodal Reasoning Models [65.57985131861399]
  hallucinations are strongly correlated with cognitive bifurcation points that often exhibit high entropy states.<n>We propose V-STAR, a lightweight, holistic training paradigm designed to internalize visually aware reasoning capabilities.
  arXiv  Detail & Related papers  (2026-04-11T13:59:05Z)
• Visually-Guided Policy Optimization for Multimodal Reasoning [60.035908460318126]
  We propose Visually-Guided Policy Optimization (VGPO) to reinforce visual focus during policy optimization.<n> VGPO initially introduces a Visual Attention Compensation mechanism that leverages visual similarity to localize and amplify visual cues.<n>We show that VGPO achieves better visual activation and superior performance in mathematical multimodal reasoning and visual-dependent tasks.
  arXiv  Detail & Related papers  (2026-04-10T14:22:38Z)
• Reflect to Inform: Boosting Multimodal Reasoning via Information-Gain-Driven Verification [55.357038267439684]
  Visual Re-Examination (VRE) is a self-evolving training framework that enables MLLMs to autonomously perform visual introspection during reasoning without additional visual inputs.<n>VRE consistently improves reasoning accuracy and perceptual reliability, while substantially reducing hallucinations, especially in long-chain settings.
  arXiv  Detail & Related papers  (2026-03-27T12:22:13Z)
• Predictive Regularization Against Visual Representation Degradation in Multimodal Large Language Models [84.94288033791346]
  We conduct a detailed diagnostic analysis to unveil a pervasive issue: visual representation degradation in MLLMs.<n>We attribute this phenomenon to a visual sacrifice driven by the singular text-generation objective, where the model compromises its visual fidelity to optimize for answer generation.<n>We propose Predictive Regularization to force degraded intermediate features to predict initial visual features, thereby maintaining the inherent visual attributes of the MLLM's internal representations.
  arXiv  Detail & Related papers  (2026-03-21T13:10:37Z)
• Insight-V++: Towards Advanced Long-Chain Visual Reasoning with Multimodal Large Language Models [65.4947731385794]
  We present a unified multi-agent visual reasoning framework that evolves from our foundational image-centric model, Insight-V.<n>We introduce two novel algorithms, ST-GRPO and J-GRPO, which enhance spatial-temporal reasoning and improve evaluative robustness.<n>Experiments on base models like LLaVA-NeXT and Qwen2.5-VL demonstrate significant performance gains across challenging image and video reasoning benchmarks.
  arXiv  Detail & Related papers  (2026-03-18T15:28:07Z)
• Stateful Cross-layer Vision Modulation [19.730096071316876]
  multimodal large language models (MLLMs) widely adopt multi-layer visual feature fusion to enhance visual representation.<n>Existing approaches typically perform static concatenation or weighted aggregation after visual encoding, without intervening in the representation formation process itself.<n>We propose a cross-layer memory-modulated vision framework(SCVM) to address these limitations.
  arXiv  Detail & Related papers  (2026-02-28T13:57:19Z)
• Revealing and Enhancing Core Visual Regions: Harnessing Internal Attention Dynamics for Hallucination Mitigation in LVLMs [67.69730908817321]
  Internal Positive Attention Dynamics (PAD) in LVLMs naturally reveal semantically core visual regions under the distortions of attention sinks.<n>We propose Positive Attention Dynamics Enhancement (PADE), a training-free attention intervention that constructs a PAD map to identify semantically core visual regions.
  arXiv  Detail & Related papers  (2026-02-17T13:08:06Z)
• Hallucination Begins Where Saliency Drops [18.189047289404325]
  hallucinations frequently arise when preceding output tokens exhibit low saliency toward the prediction of the next token.<n>We introduce LVLMs-Saliency, a gradient-aware diagnostic framework that quantifies the visual grounding strength of each output token.<n>Our method significantly reduces hallucination rates while preserving fluency and task performance, offering a robust and interpretable solution.
  arXiv  Detail & Related papers  (2026-01-28T05:50:52Z)
• GLIMPSE: Holistic Cross-Modal Explainability for Large Vision-Language Models [0.0]
  We introduce GLIMPSE, a model-agnostic framework that jointly attributes LVLM outputs to the most relevant visual evidence and textual signals.<n>GLIMPSE fuses gradient-weighted attention, adaptive layer propagation, and relevance-weighted token aggregation to produce holistic response-level heat maps.<n>We demonstrate an analytic approach to uncover fine-grained insights into LVLM cross-modal attribution, trace reasoning dynamics, analyze systematic misalignment, diagnose hallucination and bias, and ensure transparency.
  arXiv  Detail & Related papers  (2025-06-23T18:00:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.