OxyGen: Unified KV Cache Management for Vision-Language-Action Models under Multi-Task Parallelism

• URL: http://arxiv.org/abs/2603.14371v1
• Date: Sun, 15 Mar 2026 13:23:56 GMT
• Title: OxyGen: Unified KV Cache Management for Vision-Language-Action Models under Multi-Task Parallelism
• Abstract summary: We propose unified KV cache management, an inference paradigm that treats KV cache as a first-class shared resource across tasks and over time.<n> OxyGen achieves up to 3.7$times$ speedup over isolated execution, delivering over 200 tokens/s language throughput and 70 Hz action frequency simultaneously without action quality degradation.
• Score: 15.325398725305774
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Embodied AI agents increasingly require parallel execution of multiple tasks, such as manipulation, conversation, and memory construction, from shared observations under distinct time constraints. Recent Mixture-of-Transformers (MoT) Vision-Language-Action Models (VLAs) architecturally support such heterogeneous outputs, yet existing inference systems fail to achieve efficient multi-task parallelism for on-device deployment due to redundant computation and resource contention. We identify isolated KV cache management as the root cause. To address this, we propose unified KV cache management, an inference paradigm that treats KV cache as a first-class shared resource across tasks and over time. This abstraction enables two key optimizations: cross-task KV sharing eliminates redundant prefill of shared observations, while cross-frame continuous batching decouples variable-length language decoding from fixed-rate action generation across control cycles. We implement this paradigm for $π_{0.5}$, the most popular MoT VLA, and evaluate under representative robotic configurations. OxyGen achieves up to 3.7$\times$ speedup over isolated execution, delivering over 200 tokens/s language throughput and 70 Hz action frequency simultaneously without action quality degradation.

Related papers

• Training-Free Acceleration for Vision-Language-Action Models with Action Caching and Refinement [1.0279917564173053]
  Vision-Language-Action (VLA) models have emerged as a promising approach for generalizable robotic manipulations.<n>We propose ActionCache, a plug-and-play external cache that opportunistically reuses past intermediate actions to warm-start generations.<n> Experimental results in simulation and real-world environments demonstrate that ActionCache maintains high task success rates in a low-latency regime.
  arXiv  Detail & Related papers  (2026-07-07T15:10:15Z)
• Embodied.cpp: A Portable Inference Runtime of Embodied AI Models on Heterogeneous Robots [19.848719701819153]
  We present Embodied$.$, a portable runtime for embodied AI models and WAMs.<n>The runtime provides multi-rate execution, latency-first fused inference, and operator and I/O support.<n>We show that Embodied$.$ improves deployment efficiency while preserving high accuracy across diverse embodied model architectures.
  arXiv  Detail & Related papers  (2026-07-02T17:58:28Z)
• An Efficient vLLM-Based Inference Pipeline for Unified Audio Understanding and Generation [63.50393485685279]
  We present a vLLM-based inference pipeline for unified speech understanding and generation.<n>We overcome the shared intuition-Free Guidance (CFG) halves throughput.<n>By co-scheduling paired conditional and unconditional requests within a continuous batch, our CFG implementation sustains 80% of non-CFG throughput.
  arXiv  Detail & Related papers  (2026-07-02T12:55:11Z)
• OneDrive: Unified Multi-Paradigm Driving with Vision-Language-Action Models [69.2503510410147]
  We present a unified autonomous driving framework built upon a pretrained VLM.<n>We demonstrate that pretrained VLM attention exhibits strong transferability beyond pure language modeling.<n>Experiments on end-to-end autonomous driving benchmarks demonstrate state-of-the-art performance.
  arXiv  Detail & Related papers  (2026-04-20T07:50:00Z)
• Quantization with Unified Adaptive Distillation to enable multi-LoRA based one-for-all Generative Vision Models on edge [4.632054706878866]
  GenAI features such as image editing, object removal, and prompt-guided image transformation are increasingly integrated into mobile applications.<n>Existing Mobile deployment pipelines typically compile separate model binaries for each Low-Rank Adapters.<n>We present a unified framework for enabling multi-task GenAI inference on edge devices using a single shared model.
  arXiv  Detail & Related papers  (2026-03-31T10:17:07Z)
• Beyond Short-Horizon: VQ-Memory for Robust Long-Horizon Manipulation in Non-Markovian Simulation Benchmarks [96.60530830276281]
  RuleSafe is a new articulated manipulation benchmark built upon a scalable LLM-aided simulation framework.<n>VQ-Memory is a compact and structured temporal representation that uses vector-quantized variational autoencoders.
  arXiv  Detail & Related papers  (2026-03-10T11:13:54Z)
• DeltaKV: Residual-Based KV Cache Compression via Long-Range Similarity [50.52392445266824]
  We propose a residual-based KV cache compression framework motivated by long-range inter-token similarity and highly shared latent components in KV representations.<n>Instead of discarding tokens, DeltaKV encodes semantic residuals relative to retrieved historical references, preserving fidelity while substantially reducing storage.<n>Experiments show that DeltaKV reduces KV cache memory to 29% of the original while maintaining near-lossless accuracy on LongBench, SCBench, and AIME.
  arXiv  Detail & Related papers  (2026-02-08T15:14:36Z)
• Learning to Share: Selective Memory for Efficient Parallel Agentic Systems [49.78267008828593]
  Agentic systems solve complex tasks by coordinating multiple agents that iteratively reason, invoke tools, and exchange intermediate results.<n>Recent approaches deploy multiple agent teams running in parallel to explore diverse reasoning trajectories.<n>We propose Learning to Share (LTS), a learned shared-memory mechanism for parallel agentic frameworks.
  arXiv  Detail & Related papers  (2026-02-05T18:20:21Z)
• Hierarchical Adaptive Eviction for KV Cache Management in Multimodal Language Models [8.944739362562494]
  Existing KV cache eviction strategies fail to address the heterogeneous attention distributions between visual and text tokens.<n>We propose Hierarchical Adaptive Eviction (HAE), a KV cache eviction framework that optimize text-visual token interaction in MLLMs.<n>HAE minimizes KV cache usage across layers, reduces computational overhead via index broadcasting, and theoretically ensures superior information integrity and lower error bounds.
  arXiv  Detail & Related papers  (2026-02-02T15:01:44Z)
• Memory- and Latency-Constrained Inference of Large Language Models via Adaptive Split Computing [8.705453442427585]
  Large language models (LLMs) have achieved near-human performance across diverse reasoning tasks.<n>Their deployment on resource-constrained Internet-of-Things (IoT) devices remains impractical due to massive parameter footprints and memory-intensive autoregressive decoding.<n>This work introduces the first autoregressive-aware split computing framework designed explicitly for LLM deployment on edge devices.
  arXiv  Detail & Related papers  (2025-11-06T02:55:07Z)
• dVLA: Diffusion Vision-Language-Action Model with Multimodal Chain-of-Thought [66.78110237549087]
  Vision-Language-Action (VLA) models are emerging as a next-generation paradigm for robotics.<n>We introduce dVLA, a diffusion-based VLA that unifies visual perception, language reasoning, and robotic control in a single system.
  arXiv  Detail & Related papers  (2025-09-30T02:36:11Z)
• CIFLEX: Contextual Instruction Flow for Sub-task Execution in Multi-Turn Interactions with a Single On-Device LLM [25.486389526043727]
  We present a novel execution system for efficient sub-task handling in multi-turn interactions with a single on-device large language model (LLM)<n>We develop a hierarchical classification strategy tailored for small-scale models, decomposing multi-choice decisions into binary ones.<n>Experiments show that CIFLEX significantly reduces computational costs without degrading task performance, enabling scalable and efficient multi-task dialogue on-device.
  arXiv  Detail & Related papers  (2025-09-24T01:20:47Z)
• VLASCD: A Visual Language Action Model for Simultaneous Chatting and Decision Making [29.23206299246665]
  We show that the multi-input single-output (MISO) paradigm limits performance in multi-input multi-output (MIMO) scenarios.<n>In MISO architectures, tasks compete for a shared output channel, creating mutual exclusion effects that cause unbalanced optimization and degraded performance.<n>We introduce a unified training framework that enables concurrent multi-task outputs, exemplified by simultaneous dialogue generation and decision-making.
  arXiv  Detail & Related papers  (2024-10-21T11:02:42Z)
• Edge-MoE: Memory-Efficient Multi-Task Vision Transformer Architecture with Task-level Sparsity via Mixture-of-Experts [60.1586169973792]
  M$3$ViT is the latest multi-task ViT model that introduces mixture-of-experts (MoE) MoE achieves better accuracy and over 80% reduction computation but leaves challenges for efficient deployment on FPGA. Our work, dubbed Edge-MoE, solves the challenges to introduce the first end-to-end FPGA accelerator for multi-task ViT with a collection of architectural innovations.
  arXiv  Detail & Related papers  (2023-05-30T02:24:03Z)
• M$^3$ViT: Mixture-of-Experts Vision Transformer for Efficient Multi-task Learning with Model-Accelerator Co-design [95.41238363769892]
  Multi-task learning (MTL) encapsulates multiple learned tasks in a single model and often lets those tasks learn better jointly. Current MTL regimes have to activate nearly the entire model even to just execute a single task. We present a model-accelerator co-design framework to enable efficient on-device MTL.
  arXiv  Detail & Related papers  (2022-10-26T15:40:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.