KAIROS: Stateful, Context-Aware Power-Efficient Agentic Inference Serving

• URL: http://arxiv.org/abs/2604.16682v1
• Date: Fri, 17 Apr 2026 20:39:25 GMT
• Title: KAIROS: Stateful, Context-Aware Power-Efficient Agentic Inference Serving
• Abstract summary: KAIROS is a context-aware power optimization system for agentic AI serving.<n>It saves power when memory headroom exists while avoiding thrashing and preserving performance targets.<n>It achieves an average of 27% (up to 39.8%) power reduction while meeting the performance targets.
• Score: 4.817676761376181
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Power has become a central bottleneck for AI inference. This problem is becoming more urgent as agentic AI emerges as a major workload class, yet prior power-management techniques focus almost entirely on single-turn LLM serving. Our analysis shows that agentic serving behaves fundamentally differently: each request carries long-lived context that evolves across tool-interleaved turns, and lowering GPU frequency can push the system into a thrashing regime where memory pressure sharply worsens both performance and power efficiency. These observations show that power optimization for agentic serving requires rethinking. We present KAIROS, a context-aware power optimization system for agentic AI serving. KAIROS uses agent context as a first-class control signal to jointly manage GPU frequency, per-instance concurrency, and multi-instance request placement. This enables KAIROS to save power when memory headroom exists while avoiding thrashing and preserving performance targets. At a high level, KAIROS tracks requests at agent granularity, adapts local control to context growth and agent progress, and routes agents across instances to jointly improve power efficiency and memory stability. Evaluated across diverse software and data engineering agentic tasks, KAIROS achieves an average of 27% (up to 39.8%) power reduction while meeting the performance targets.

Related papers

• MARS: Efficient, Adaptive Co-Scheduling for Heterogeneous Agentic Systems [22.470203759341004]
  Large language models (LLMs) are increasingly deployed as the execution core of autonomous agents.<n> coordinating heterogeneous resource demands of agentic execution has emerged as a critical system challenge.<n>We design and implement MARS, an efficient and adaptive co-scheduling system.<n>Our evaluations show that MARS reduces end-to-end latency by up to 5.94x while maintaining nearly maximal system throughput.
  arXiv  Detail & Related papers  (2026-04-14T05:15:28Z)
• When Should a Robot Think? Resource-Aware Reasoning via Reinforcement Learning for Embodied Robotic Decision-Making [68.12864562049957]
  Embodied robotic systems increasingly rely on large language model (LLM)-based agents to support high-level reasoning.<n>This raises a fundamental question for embodied agents: when should the agent reason, and when should it act?<n>We propose RARRL (Resource-Aware Reasoning via Reinforcement Learning), a hierarchical framework for resource-aware orchestration of embodied agents.
  arXiv  Detail & Related papers  (2026-03-17T15:38:50Z)
• AgentCgroup: Understanding and Controlling OS Resources of AI Agents [2.8139711959925244]
  AI agents are increasingly deployed in multi-tenant cloud environments, where they execute diverse tool calls within sandboxed containers.<n>We present a systematic characterization of OS-level resource dynamics in sandboxed AI coding agents.<n>Preliminary evaluation demonstrates improved multi-tenant isolation and reduced resource waste.
  arXiv  Detail & Related papers  (2026-02-10T02:37:42Z)
• Learning to Share: Selective Memory for Efficient Parallel Agentic Systems [49.78267008828593]
  Agentic systems solve complex tasks by coordinating multiple agents that iteratively reason, invoke tools, and exchange intermediate results.<n>Recent approaches deploy multiple agent teams running in parallel to explore diverse reasoning trajectories.<n>We propose Learning to Share (LTS), a learned shared-memory mechanism for parallel agentic frameworks.
  arXiv  Detail & Related papers  (2026-02-05T18:20:21Z)
• AgentArk: Distilling Multi-Agent Intelligence into a Single LLM Agent [57.10083973844841]
  AgentArk is a novel framework to distill multi-agent dynamics into the weights of a single model.<n>We investigate three hierarchical distillation strategies across various models, tasks, scaling, and scenarios.<n>By shifting the burden of computation from inference to training, the distilled models preserve the efficiency of one agent while exhibiting strong reasoning and self-correction performance of multiple agents.
  arXiv  Detail & Related papers  (2026-02-03T19:18:28Z)
• SCOPE: Prompt Evolution for Enhancing Agent Effectiveness [53.75986399936395]
  Large Language Model (LLM) agents are increasingly deployed in environments that generate massive, dynamic contexts.<n>While agents have access to this context, their static prompts lack the mechanisms to manage it effectively.<n>We introduce textbfSCOPE (Self-evolving Context Optimization via Prompt Evolution)<n>We propose a Dual-Stream mechanism that balances tactical specificity (resolving immediate errors) with strategic generality (evolving long-term principles)
  arXiv  Detail & Related papers  (2025-12-17T12:25:05Z)
• Astraea: A State-Aware Scheduling Engine for LLM-Powered Agents [12.884297990127985]
  Astraea is a service engine designed to shift the optimization from local segments to the global request lifecycle.<n>It employs a state-aware, hierarchical scheduling algorithm that integrates a request's historical state with future predictions.<n>Astraea reduces average JCT by up to 25.5% compared to baseline methods.
  arXiv  Detail & Related papers  (2025-12-16T06:55:10Z)
• A CPU-Centric Perspective on Agentic AI [8.417523196411574]
  Agentic AI frameworks add a decision-making orchestrator embedded with external tools, including web search, Python interpreter, contextual database, and others.<n>This paper aims to characterize and understand the system bottlenecks introduced by agentic AI workloads from a CPU-centric perspective.
  arXiv  Detail & Related papers  (2025-11-01T23:46:44Z)
• Hi-Agent: Hierarchical Vision-Language Agents for Mobile Device Control [72.43808515668947]
  We introduce Hi-Agent, a trainable hierarchical vision-language agent for mobile control.<n>Hi-Agent features a high-level reasoning model and a low-level action model that are jointly optimized.<n>Hi-Agent achieves a new State-Of-The-Art (SOTA) 87.9% task success rate on the Android-in-the-Wild (AitW) benchmark.
  arXiv  Detail & Related papers  (2025-10-16T07:38:21Z)
• Agent.xpu: Efficient Scheduling of Agentic LLM Workloads on Heterogeneous SoC [11.82567747365518]
  This paper introduces Agent.xpu, an efficient serving system for agentic LLM workloads on memory-unified heterogeneous SOCs.<n>With dedicated offline profiling, Agent.xpu first constructs a heterogeneous execution graph, which fuses and chunks model kernels for affinity-guided, elastic accelerator mapping.<n>At runtime, its online scheduler enables fine-grained, kernel-level preemption to guarantee the responsiveness of reactive tasks.
  arXiv  Detail & Related papers  (2025-06-30T16:50:48Z)
• Distributed-Training-and-Execution Multi-Agent Reinforcement Learning for Power Control in HetNet [48.96004919910818]
  We propose a multi-agent deep reinforcement learning (MADRL) based power control scheme for the HetNet. To promote cooperation among agents, we develop a penalty-based Q learning (PQL) algorithm for MADRL systems. In this way, an agent's policy can be learned by other agents more easily, resulting in a more efficient collaboration process.
  arXiv  Detail & Related papers  (2022-12-15T17:01:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.