Anatomy of a Quantized Agent: VRAM Stability and Forecasting in Code-Synthesis Agentic Workloads

• URL: http://arxiv.org/abs/2608.15117v1
• Date: Sat, 15 Aug 2026 08:39:43 GMT
• Title: Anatomy of a Quantized Agent: VRAM Stability and Forecasting in Code-Synthesis Agentic Workloads
• Abstract summary: Analytical models of peak VRAM consumption for LLM inference decompose memory into weight-storage, KV-cache, and activation terms.<n>We evaluate this decomposition empirically within a strictly scoped measurement study.<n>We find no justification for deploying complex predictive VRAM models in highly quantized, weight-dominated regimes.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Analytical models of peak VRAM consumption for LLM inference decompose memory into weight-storage, KV-cache, and activation terms parameterized by step count, tool invocations, and context expansion. We evaluate this decomposition empirically within a strictly scoped measurement study: a LangGraph-based CUDA-kernel-synthesis agent (AgentK), a 4-bit quantization family (Q4 K M), a single NVIDIA H100 GPU, and four LLM backbones across 1,920 trajectories. Focusing on peak-memory forecasting behavior, we report two primary observations. First, closed-form analytical models achieve competitive accuracy when provided with two empirical constants: loaded-weight VRAM and a fixed activation-memory overhead. Supplied with live GPU readings and ground-truth trajectory parameters, the closed-form model matches or outperforms the best learned baseline on three of the four backbones (test MAPE 2.2-4.4% vs. 3.4-6.5%, p = 0.76). The exception is the smallest backbone (Phi-4-mini), where minimal VRAM variance (CV 0.3%) causes dynamic modeling to underperform simple regression. Second, compile success strictly bifurcates by backbone capacity (from 5.7% for Phi-4-mini to 62.0% for Qwen2.5-Coder-14B), demonstrating that functional code synthesis remains constrained by intrinsic LLM capabilities rather than available memory. Furthermore, because overall peak-memory variance is remarkably low across all backbones (CV 0.3-9.4%), learned prompt-feature regression offers statistically insignificant improvements over a constant-mean baseline. Consequently, we find no justification for deploying complex predictive VRAM models in highly quantized, weight-dominated regimes. We release the evaluated corpus and anonymized framework to support replication.

Related papers

• TriCalRAG: A Three-Strategy, Retrieval-Augmented Benchmark for On-Premise LLM-Based Root Cause Analysis in AIOps [1.8136615181861766]
  Cloud-hosted large language models (LLMs) are increasingly used for root cause analysis (RCA) in AIOps pipelines.<n>They introduce data privacy risk, network latency, and per-query cost that scale poorly with production log volumes.<n>We presentRAG, a benchmark evaluating open-weights served locally via vLLM against a classical LSTM-based log anomaly detector.
  arXiv  Detail & Related papers  (2026-09-13T19:52:46Z)
• SLAI T-Rex: Full-Parameter Post-training of the DeepSeek-V4 Family on Ascend SuperPOD [91.71050339479261]
  This report presents an end-to-end optimization practice on the Ascend NPU SuperPOD.<n>We develop a hierarchical optimization framework spanning model-level parallelism, computation-communication orchestration, and low-level kernel execution.<n>The resulting system achieves 34.22% Model FLOPs Utilization (MFU) with a 2.93x improvement over the open-source baseline recipe.
  arXiv  Detail & Related papers  (2026-07-22T13:49:17Z)
• $μ$VLA: On Recurrent Memory for Partially Observable Manipulation in VLA Models [44.66763491853812]
  We present a controlled isolation study of recurrence in a strong pretrained VLA backbone.<n>Our formulation augments the transformer with a small set of learnable memory tokens carried across timesteps.<n>We interpret these results as a calibration of the capability envelope of minimal in-backbone recurrence.
  arXiv  Detail & Related papers  (2026-06-10T13:26:40Z)
• FlashMemory-DeepSeek-V4: Lightning Index Ultra-Long Context via Lookahead Sparse Attention [77.12062766962815]
  Lookahead Sparse Attention (LSA) is powered by a Neural Memory Indexer built upon the DeepSeek-V4 architecture.<n>We instantiate this architecture via a backbone-free decoupled training strategy.<n>We show that FM-DS-V4 compresses the average physical KV cache footprint down to merely 13.5% of the full-context baseline.
  arXiv  Detail & Related papers  (2026-06-08T06:25:54Z)
• MEMTIER: Tiered Memory Architecture and Retrieval Bottleneck Analysis for Long-Running Autonomous AI Agents [6.8552591401084015]
  We present MEMTIER, a tripartite memory architecture for the OpenClaw agent.<n>On the full LongMemEval-S benchmark, MEMTIER achieves Acc.382, F1=.412 with Qwen2.5-7B on a consumer 6GB GPU.<n>With DeepSeek-V4-Flash fact pre-population, single-session recall reaches 0.686-0.714, exceeding the paper's RAG GPT-4o on those categories.
  arXiv  Detail & Related papers  (2026-05-05T12:14:10Z)
• Prism: An Evolutionary Memory Substrate for Multi-Agent Open-Ended Discovery [0.0]
  We introduce prism (textbfProbabilistic replicatextbfRetrieval with textbfInformation-textbfStratified textbfMemory), an evolutionary memory substrate for multi-agent AI systems.<n>prism unifies four independently developed paradigms -- layered file-based persistence, vector-augmented semantic memory, graph-structured relational memory, and multi-agent evolutionary search.
  arXiv  Detail & Related papers  (2026-04-08T09:16:43Z)
• Choosing the Right Regularizer for Applied ML: Simulation Benchmarks of Popular Scikit-learn Regularization Frameworks [0.1862063702912854]
  We empirically evaluate the performance of four canonical frameworks -- Ridge, Lasso, ElasticNet, and Post-Lasso OLS -- across 134,400 simulations.<n>For prediction accuracy when the sample-to-feature ratio is sufficient, Ridge, Lasso, and ElasticNet are nearly interchangeable.<n>We advise practitioners against using Lasso or Post-Lasso OLS at high kappa with small sample sizes.
  arXiv  Detail & Related papers  (2026-04-04T01:33:41Z)
• Optimized Minimal 4D Gaussian Splatting [50.29519144753797]
  4D Gaussian Splatting has emerged as a new paradigm for dynamic scene representation, enabling real-time rendering of scenes with complex motions.<n>It faces a major challenge of storage overhead, as millions of Gaussians are required for high-fidelity reconstruction.<n>We present OMG4, a framework that constructs a compact set of salient Gaussians capable of faithfully representing 4D Gaussian models.
  arXiv  Detail & Related papers  (2025-10-04T16:11:13Z)
• EfficientLLM: Efficiency in Large Language Models [64.3537131208038]
  Large Language Models (LLMs) have driven significant progress, yet their growing counts and context windows incur prohibitive compute, energy, and monetary costs.<n>We introduce EfficientLLM, a novel benchmark and the first comprehensive empirical study evaluating efficiency techniques for LLMs at scale.
  arXiv  Detail & Related papers  (2025-05-20T02:27:08Z)
• SqueezeLLM: Dense-and-Sparse Quantization [80.32162537942138]
  Main bottleneck for generative inference with LLMs is memory bandwidth, rather than compute, for single batch inference. We introduce SqueezeLLM, a post-training quantization framework that enables lossless compression to ultra-low precisions of up to 3-bit. Our framework incorporates two novel ideas: (i) sensitivity-based non-uniform quantization, which searches for the optimal bit precision assignment based on second-order information; and (ii) the Dense-and-Sparse decomposition that stores outliers and sensitive weight values in an efficient sparse format.
  arXiv  Detail & Related papers  (2023-06-13T08:57:54Z)
• Inception Convolution with Efficient Dilation Search [121.41030859447487]
  Dilation convolution is a critical mutant of standard convolution neural network to control effective receptive fields and handle large scale variance of objects. We propose a new mutant of dilated convolution, namely inception (dilated) convolution where the convolutions have independent dilation among different axes, channels and layers. We explore a practical method for fitting the complex inception convolution to the data, a simple while effective dilation search algorithm(EDO) based on statistical optimization is developed.
  arXiv  Detail & Related papers  (2020-12-25T14:58:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.