OOM-Free Alpamayo via CPU-GPU Memory Swapping for Vision-Language-Action Models

• URL: http://arxiv.org/abs/2605.11678v1
• Date: Tue, 12 May 2026 07:37:10 GMT
• Title: OOM-Free Alpamayo via CPU-GPU Memory Swapping for Vision-Language-Action Models
• Abstract summary: Vision-Language-Action (VLA) models for autonomous driving unify perception, reasoning, and control in a single neural network.<n>We present a framework, which enables memory-efficient VLA inference on VRAM-constrained GPU.<n>Our work achieves up to 3.55x speedup over Accelerate offloading while maintaining full BF16 precision.
• Score: 1.1011268090482575
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: End-to-end Vision-Language-Action (VLA) models for autonomous driving unify perception, reasoning, and control in a single neural network, achieving strong driving performance but requiring 20-60GB of GPU memory-far exceeding the 12-16GB available on commodity GPUs. We present a framework, which enables memory-efficient VLA inference on VRAM-constrained GPUs through system-level optimization alone, without model modification. Our work proceeds in three stages: (1) Sequential Demand Layering reduces VRAM usage from model-level to layer-level granularity; (2) Pipelined Demand Layering hides parameter transfer time within layer execution time via transfer--compute overlap; and (3) a GPU-Resident Layer Decision Policy, informed by per-module residency benefit analysis, eliminates the residual transfer overhead that pipelining cannot hide. We further propose a performance prediction model that determines the optimal configuration-both the number and placement of resident layers-from a single profiling run with less than 1.3% prediction error across all configurations. Applied to NVIDIA's Alpamayo-R1-10B (21.52GB) on an RTX 5070Ti (16GB), our work achieves up to 3.55x speedup over Accelerate offloading while maintaining full BF16 precision.

Related papers

• MegaSlide-DiT: Memory-Centric Adaptation and Deformable Local Attention for Efficient Video Diffusion [3.7821833028870038]
  MegaSlide-DiT is a prototype that demonstrates how a pre-trained 105B DiT can be adapted on a single H200 GPU with 1.5 TB of host RAM.<n>Our own insight is that not all persistent weights, master weights and moments remain in host memory, while only transient shards are streamed to the GPU on demand.
  arXiv  Detail & Related papers  (2026-07-17T19:30:05Z)
• Efficient, VRAM-Constrained xLM Inference on Clients [0.0]
  We present pipelined sharding, a novel, benchmark-profile-guided CPU-GPU hybrid scheduling technique.<n>It achieves efficient, VRAM-constrained inference for both dense and mixture-of-experts (MoE) large language models (LLMs) on client systems.<n>This paper is accepted at the 9th MLSys Conference (Industry Track), 2026.
  arXiv  Detail & Related papers  (2026-04-29T06:35:35Z)
• An Efficient Heterogeneous Co-Design for Fine-Tuning on a Single GPU [10.811837575220814]
  Fine-tuning Large Language Models (LLMs) has become essential for domain adaptation, but its memory-intensive property exceeds the capabilities of most GPU.<n>We present SlideFormer, a novel system designed for single-GPU environments.<n>A lightweight asynchronous engine that treats the GPU as a sliding window and overlaps GPU with CPU updates and multi-tier I/O.
  arXiv  Detail & Related papers  (2026-03-17T12:05:17Z)
• LODGE: Level-of-Detail Large-Scale Gaussian Splatting with Efficient Rendering [75.67501939005119]
  We present a novel level-of-detail (LOD) method for 3D Gaussian Splatting on memory-constrained devices.<n>Our approach iteratively selects optimal subsets of Gaussians based on camera distance.<n>Our method achieves state-of-the-art performance on both outdoor (Hierarchical 3DGS) and indoor (Zip-NeRF) datasets.
  arXiv  Detail & Related papers  (2025-05-29T06:50:57Z)
• ZO2: Scalable Zeroth-Order Fine-Tuning for Extremely Large Language Models with Limited GPU Memory [29.245719403159615]
  We propose a novel framework, ZO2, for efficient zeroth-order fine-tuning of LLMs with only limited GPU memory.<n>Our framework supports an innovative low-bit precision approach in AMP mode to streamline data exchanges between the CPU and GPU.
  arXiv  Detail & Related papers  (2025-03-16T21:58:29Z)
• HeadInfer: Memory-Efficient LLM Inference by Head-wise Offloading [79.38548165722229]
  HEADINFER offloads the KV cache to CPU RAM while avoiding the need to fully store the KV cache for any transformer layer on the GPU.<n>We demonstrate HEADINFER maintains computational efficiency while significantly reducing memory footprint.
  arXiv  Detail & Related papers  (2025-02-18T06:26:05Z)
• APOLLO: SGD-like Memory, AdamW-level Performance [61.53444035835778]
  Large language models (LLMs) are notoriously memory-intensive during training.<n>Various memory-efficient Scals have been proposed to reduce memory usage.<n>They face critical challenges: (i) costly SVD operations; (ii) significant performance trade-offs compared to AdamW; and (iii) still substantial memory overhead to maintain competitive performance.
  arXiv  Detail & Related papers  (2024-12-06T18:55:34Z)
• Pushing the Performance Envelope of DNN-based Recommendation Systems Inference on GPUs [13.720423381263409]
  We show that the embedding stage continues to be the primary bottleneck in the GPU inference pipeline, leading to a 3.2x embedding-only performance slowdown. We propose specialized plug-and-play-based software prefetching and L2 pinning techniques, which help in hiding and decreasing the latencies. Our proposed techniques improve performance by up to 103% for the embedding stage, and up to 77% for the overall DLRM inference pipeline.
  arXiv  Detail & Related papers  (2024-10-29T17:13:54Z)
• vTensor: Flexible Virtual Tensor Management for Efficient LLM Serving [53.972175896814505]
  Large Language Models (LLMs) are widely used across various domains, processing millions of daily requests. Large Language Models (LLMs) are widely used across various domains, processing millions of daily requests.
  arXiv  Detail & Related papers  (2024-07-22T14:37:58Z)
• Practical offloading for fine-tuning LLM on commodity GPU via learned sparse projectors [11.127604539303373]
  Fine-tuning large language models (LLMs) requires significant memory, often exceeding the capacity of a single GPU.<n>A common solution to this memory challenge is offloading compute and data from the GPU to the CPU.<n>We present an offloading framework, LSP-Offload, that enables near-native speed LLM fine-tuning on commodity hardware.
  arXiv  Detail & Related papers  (2024-06-14T16:59:11Z)
• An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Models [65.37846460916042]
  We find out that the attention computation over visual tokens is of extreme inefficiency in the deep layers of popular LVLMs. We introduce FastV, a versatile plug-and-play method designed to optimize computational efficiency.
  arXiv  Detail & Related papers  (2024-03-11T14:35:32Z)
• FusionAI: Decentralized Training and Deploying LLMs with Massive Consumer-Level GPUs [57.12856172329322]
  We envision a decentralized system unlocking the potential vast untapped consumer-level GPU. This system faces critical challenges, including limited CPU and GPU memory, low network bandwidth, the variability of peer and device heterogeneity.
  arXiv  Detail & Related papers  (2023-09-03T13:27:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.