Dynamic Context Adapters: Efficiently Infusing History into Vision-and-Language Models

• URL: http://arxiv.org/abs/2608.10525v1
• Date: Tue, 11 Aug 2026 06:05:05 GMT
• Title: Dynamic Context Adapters: Efficiently Infusing History into Vision-and-Language Models
• Abstract summary: We introduce Dynamic Context Adapter (DCA), a novel context injection approach for pretrained Vision-Language Models (VLMs)<n>Our method employs fixed-size, dynamically compressed memory to preserve historical semantics without frame concatenation.<n>DCA achieves over $25%$ reduction in attention FLOPs and $13%$ memory savings while improving performance on long-horizon tasks.
• Score: 19.861339694649256
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Historical context integration presents a fundamental challenge for Vision-Language Models (VLMs) in sequential decision-making tasks. Current VLMs process visual inputs independently, which creates critical limitations for downstream applications that require temporal understanding. Direct incorporation of historical frames into Transformer inputs produces quadratic attention complexity and excessive memory consumption. Existing approaches suffer from significant drawbacks: computational inflation or substantial information loss through temporal compression. To address these challenges, we introduce Dynamic Context Adapter (DCA), a novel context injection approach for pretrained VLMs. Our method employs fixed-size, dynamically compressed memory to preserve historical semantics without frame concatenation. DCA bridges static VLMs and recurrent policies and enables memory capabilities in pretrained models while maintaining computational efficiency. DCA achieves over $25\%$ reduction in attention FLOPs and $13\%$ memory savings while improving performance on long-horizon tasks.

Related papers

• MemoryWAM: Efficient World Action Modeling with Persistent Memory [80.90899269062128]
  We introduce MemoryWAM, a world action model with efficient persistent memory.<n>A tailored attention mechanism enables retrieval of both detailed short-term context and compressed long-term context.<n>MemoryWAM outperforms strong vision-language-action (VLA) and WAM baselines in both simulation and the real world.
  arXiv  Detail & Related papers  (2026-06-18T17:59:51Z)
• EventVLA: Event-Driven Visual Evidence Memory for Long-Horizon Vision-Language-Action Policies [68.812675280427]
  EventVLA is an end-to-end framework founded on the concept of sparse visual evidence memory.<n>KEM directly predicts future probabilities from the VLA's latent embeddings to autonomously capture and store sparse, task-critical visual events.<n>We propose RoboTwin-MeM, a diagnostic benchmark designed to evaluate non-Markovian manipulation tasks with interactive visual evidence.
  arXiv  Detail & Related papers  (2026-06-18T11:11:37Z)
• FLARE: Diffusion for Hybrid Language Model [72.60770374799634]
  FLARE is a systematic conversion framework for hybrid-attention large language models.<n>It combines a token-equal AR-and-diffusion objective, hardware-aware kernels, and unified inference, enabling one checkpoint to support both AR-style verified decoding and diffusion-style parallel denoising.<n>Our results suggest practical dLLMs are limited not only by decoding algorithms, but also by transfer data quality and the training inefficiency of current block-diffusion objectives.
  arXiv  Detail & Related papers  (2026-06-01T06:58:15Z)
• Uni-MDTrack: Learning Decoupled Memory and Dynamic States for Parameter-Efficient Visual Tracking in All Modality [44.07279162081132]
  We propose Uni-MDTrack, which consists of two core components: Memory-Aware Compression Prompt (MCP) module and Dynamic State Fusion (DSF) module.<n>MCP effectively compresses rich memory features into memory-aware prompt tokens, which deeply interact with the input throughout the backbone.<n>DSF also introduces updated dynamic state features from shallow to deep layers, while also preserving high efficiency.
  arXiv  Detail & Related papers  (2026-03-15T15:56:41Z)
• Beyond Short-Horizon: VQ-Memory for Robust Long-Horizon Manipulation in Non-Markovian Simulation Benchmarks [96.60530830276281]
  RuleSafe is a new articulated manipulation benchmark built upon a scalable LLM-aided simulation framework.<n>VQ-Memory is a compact and structured temporal representation that uses vector-quantized variational autoencoders.
  arXiv  Detail & Related papers  (2026-03-10T11:13:54Z)
• Memo: Training Memory-Efficient Embodied Agents with Reinforcement Learning [53.72709564555407]
  Memo is a transformer-based architecture and training recipe for reinforcement learning.<n>It incorporates the creation and retrieval of memory by interleaving periodic summarization tokens with the inputs of a model during training.<n>We demonstrate Memo's effectiveness on a gridworld meta-RL benchmark and a multi-object navigation task in photo-realistic indoor settings.
  arXiv  Detail & Related papers  (2025-10-22T16:24:47Z)
• DAF: An Efficient End-to-End Dynamic Activation Framework for on-Device DNN Training [41.09085549544767]
  We introduce a Dynamic Activation Framework (DAF) that enables scalable and efficient on-device training through system-level optimizations.<n>DAF achieves both memory- and time-efficient dynamic quantization training by addressing key system bottlenecks.<n> Evaluations on various deep learning models across embedded and mobile platforms demonstrate up to a $22.9times$ reduction in memory usage and a $3.2times$ speedup.
  arXiv  Detail & Related papers  (2025-07-09T08:59:30Z)
• Make It Efficient: Dynamic Sparse Attention for Autoregressive Image Generation [8.624395048491275]
  We propose a novel training-free context optimization method called Adaptive Dynamic Sparse Attention (ADSA)<n>ADSA identifies historical tokens crucial for maintaining local texture consistency and those essential for ensuring global semantic coherence, thereby efficiently streamlining attention.<n>We also introduce a dynamic KV-cache update mechanism tailored for ADSA, reducing GPU memory consumption during inference by approximately $50%$.
  arXiv  Detail & Related papers  (2025-06-23T01:27:06Z)
• Shifting AI Efficiency From Model-Centric to Data-Centric Compression [67.45087283924732]
  We argue that the focus of research for AI is shifting from model-centric compression to data-centric compression.<n>Data-centric compression improves AI efficiency by directly compressing the volume of data processed during model training or inference.<n>Our work aims to provide a novel perspective on AI efficiency, synthesize existing efforts, and catalyze innovation to address the challenges posed by ever-increasing context lengths.
  arXiv  Detail & Related papers  (2025-05-25T13:51:17Z)
• Dynamic-LLaVA: Efficient Multimodal Large Language Models via Dynamic Vision-language Context Sparsification [27.84464349480918]
  We propose Dynamic vision-language context sparsification framework Dynamic-LLaVA.<n>It dynamically reduces the redundancy of vision context in the prefill stage.<n>It decreases the memory and computation overhead of the generated language context during decoding.
  arXiv  Detail & Related papers  (2024-12-01T16:32:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.