InduceKV: Fixed-Footprint Continual Adaptation of Multimodal LLMs via Inducing KV Memories

• URL: http://arxiv.org/abs/2607.02010v1
• Date: Thu, 02 Jul 2026 10:45:28 GMT
• Title: InduceKV: Fixed-Footprint Continual Adaptation of Multimodal LLMs via Inducing KV Memories
• Abstract summary: InduceKV is a retrieval-based method that stores each selected training prefix as an attention-ready memory entry.<n>We show that InduceKV consistently improves over PEFT, MoE, replay, and prompt-retrieval baselines under matched memory budgets.
• Score: 6.141047838371132
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multimodal large language models must adapt to evolving tasks and domains, yet continual improvement under bounded deployment footprint remains difficult because repeated parameter updates or growing replay stores can accumulate adaptation state over time. We study fixed-footprint continual adaptation: the deployed adaptation state is kept under a fixed memory budget, while the backbone model is left unchanged and task-specific updates are externalized. We propose InduceKV, a retrieval-based method that stores each selected training prefix as an attention-ready memory entry, consisting of a frozen retrieval key and compact layerwise key--value (KV) payloads that can be appended to the model's self-attention cache. Under a strict memory budget, InduceKV constructs a compact inducing set through bilevel selection: a lightweight calibration is fit for retrieval, while the selected memory balances current-task likelihood, anchor-based retention, and coverage in the frozen retrieval space. Across task-incremental instruction tuning, continual VQA, domain-incremental adaptation, and lifelong multimodal instruction tuning, InduceKV consistently improves over PEFT, MoE, replay, and prompt-retrieval baselines under matched memory budgets. We further report backbone-matched, stage-1 CoIN, compute-matched, and scalability diagnostics, showing that the gains are not due to a stronger backbone, replay alone, or an unbounded candidate pool.

Related papers

• GrowPage: On-Demand KV Budgeting for Efficient LLM Reasoning Serving [17.04616426989979]
  Long-output reasoning has made the key--value ( KV) cache a critical memory bottleneck for efficient serving.<n>Existing KV compression methods usually rely on a predefined per-request budget and adjust only which KV states are retained, leaving the total capacity fixed throughout decoding.<n>We introduce GrowPage, an on-demand KV budgeting framework that treats KV capacity as a runtime resource.
  arXiv  Detail & Related papers  (2026-09-03T07:53:05Z)
• $μ$VLA: On Recurrent Memory for Partially Observable Manipulation in VLA Models [44.66763491853812]
  We present a controlled isolation study of recurrence in a strong pretrained VLA backbone.<n>Our formulation augments the transformer with a small set of learnable memory tokens carried across timesteps.<n>We interpret these results as a calibration of the capability envelope of minimal in-backbone recurrence.
  arXiv  Detail & Related papers  (2026-06-10T13:26:40Z)
• MIITA: Memory-Induced Inference-Time Adaptation for Continual Learning with Small Language Models [64.2820121997882]
  MIITA is a Memory-Induced Inference-Time Adaptation framework for supervised CL under constrained storage.<n>It stores supervised experiences as compact correction-direction prototypes with semantic anchors, and retrieves them at inference time using semantic and uncertainty-based cues.<n>Experiments show that MIITA consistently improves final performance and mitigates forgetting under fixed memory budgets.
  arXiv  Detail & Related papers  (2026-05-20T03:03:04Z)
• GoTTA be Diverse: Rethinking Memory Policies for Test-Time Adaptation [54.70334474895361]
  Test-time adaptation (TTA) enables a pre-trained model to adapt online to an unlabeled test stream under distribution shift.<n>While most TTA research focuses on the adaptation objective, practical streams also depend critically on the memory used to select which test samples drive adaptation.<n>We introduce Guided Observational Test-Time Adaptation (GOTTA), a family of diversity-aware memory policies that combine class-balanced allocation with feature-space diversity.
  arXiv  Detail & Related papers  (2026-05-19T14:18:46Z)
• Improving the Performance and Learning Stability of Parallelizable RNNs Designed for Ultra-Low Power Applications [1.226598527858578]
  Bistable Memory Recurrent Unit (BMRU) was introduced to enable hardware-software co-design of ultra-low power RNNs.<n>BMRU performance lags behind parallelizable RNNs on complex sequential tasks.<n>We propose a cumulative update formulation that restores flow while preserving persistent memory, creating skip-connections through time.
  arXiv  Detail & Related papers  (2026-05-12T09:39:33Z)
• A Parametric Memory Head for Continual Generative Retrieval [52.66674234249913]
  Generative information retrieval (GenIR) consolidates retrieval into a single neural model that decodes document identifiers (docids) directly from queries.<n>We show that sequential adaptation improves retrieval on newly added documents but substantially degrades performance on earlier slices.<n>We propose post-adaptation memory tuning (PAMT), a memory-only stabilization stage that augments an adapted model with a modular parametric memory head.
  arXiv  Detail & Related papers  (2026-04-25T17:38:51Z)
• MemDLM: Memory-Enhanced DLM Training [56.40248490616793]
  Diffusion Language Models (DLMs) offer attractive advantages over Auto-Regressive (AR) models.<n>They suffer from a notable train-inference mismatch.<n>We propose MemDLM (Memory-Enhanced DLM), which embeds a simulated denoising process into training.
  arXiv  Detail & Related papers  (2026-03-23T17:39:56Z)
• TS-Memory: Plug-and-Play Memory for Time Series Foundation Models [63.21390142212087]
  Time Series Foundation Models (TSFMs) achieve strong zero-shot forecasting through large-scale pre-training.<n>Existing solutions face a trade-off: Parametric Adaptation can cause catastrophic forgetting, while Non-Parametric Retrieval improves forecasts but incurs high latency due to datastore search.<n>We propose Parametric Memory Distillation and implement it as TS-Memory, a lightweight memory adapter that augments frozen TSFMs.
  arXiv  Detail & Related papers  (2026-02-12T04:16:19Z)
• Training-free Context-adaptive Attention for Efficient Long Context Modeling [57.703159205740185]
  Training-free Context-adaptive Attention (TCA-Attention) is a training-free sparse attention mechanism that selectively attends to only the informative tokens for efficient long-context inference.<n>TCA-Attention achieves a 2.8$times$ speedup and reduces KV cache by 61% at 128K context length while maintaining performance comparable to full attention.
  arXiv  Detail & Related papers  (2025-12-10T01:54:57Z)
• Lethe: Layer- and Time-Adaptive KV Cache Pruning for Reasoning-Intensive LLM Serving [11.750209684686707]
  Generative reasoning with large language models (LLMs) often involves long decoding sequences.<n>We propose Lethe, a dynamic KV cache management framework that introduces adaptivity along both the spatial and temporal dimensions of decoding.<n>Lethe achieves a favorable balance between efficiency and generation quality across diverse models and tasks, increases throughput by up to 2.56x.
  arXiv  Detail & Related papers  (2025-11-08T14:52:43Z)
• LoRC: Low-Rank Compression for LLMs KV Cache with a Progressive Compression Strategy [59.1298692559785]
  Key-Value ( KV) cache is crucial component in serving transformer-based autoregressive large language models (LLMs) Existing approaches to mitigate this issue include: (1) efficient attention variants integrated in upcycling stages; (2) KV cache compression at test time; and (3) KV cache compression at test time. We propose a low-rank approximation of KV weight matrices, allowing plug-in integration with existing transformer-based LLMs without model retraining. Our method is designed to function without model tuning in upcycling stages or task-specific profiling in test stages.
  arXiv  Detail & Related papers  (2024-10-04T03:10:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.