Energy-Efficient LLM Serving via Disaggregated Attention--FFN and Flexible Frequency Scaling

• URL: http://arxiv.org/abs/2608.01891v1
• Date: Mon, 03 Aug 2026 08:32:50 GMT
• Title: Energy-Efficient LLM Serving via Disaggregated Attention--FFN and Flexible Frequency Scaling
• Abstract summary: Existing energy-management approaches adapt only at the request or inference-phase level.<n>We present AFlex, a framework that jointly optimize resource provisioning and GPU frequency scaling for disaggregated A/F serving.<n>AFlex reduces energy per token by up to 49% over state-of-the-art disaggregated serving and 48% over frequency-scaling systems.
• Score: 7.937386108138415
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language model (LLM) serving spans diverse applications with stringent service-level objectives (SLOs), often requiring GPUs to run at maximum frequencies and increasing energy consumption. Existing energy-management approaches adapt GPU frequencies only at the request or inference-phase level, overlooking operator-level differences in frequency sensitivity between Attention and feed-forward networks (FFNs). We find that the energy-optimal frequencies of Attention and FFN (A/F) differ and vary with the inference phase, workload, and system configurations. However, runtime variability and independent A/F frequency control create a large search space and high communication overhead. To address these challenges, we present AFlex, a framework that jointly optimizes resource provisioning and GPU frequency scaling for disaggregated A/F serving. AFlex introduces a global scheduler and a local operator-level dynamic voltage and frequency scaling (DVFS) controller to determine A/F resource allocations and frequencies. It further introduces an interleaved A/F pipeline with dynamic microbatch depth and adaptive request batching to reduce pipeline bubbles. We implement AFlex in SGLang and evaluate it on NVIDIA A800 GPUs using Qwen3-32B and Mixtral-8$\times$7B under production Conversation and Coding traces. \AFlex reduces energy per token by up to 49\% over state-of-the-art disaggregated serving and 48\% over frequency-scaling systems while satisfying TTFT and TPOT SLOs.

Related papers

• FreqFormer: Hierarchical Frequency-Domain Attention with Adaptive Spectral Routing for Long-Sequence Video Diffusion Transformers [0.0]
  We present FreqFormer, a frequency-aware heterogeneous attention framework.<n> Token features are split into spectral bands with different operators.<n>FreqFormer substantially reduces estimated attention FLOPs and KV-related memory traffic versus dense attention.
  arXiv  Detail & Related papers  (2026-04-14T08:29:10Z)
• SparseDVFS: Sparse-Aware DVFS for Energy-Efficient Edge Inference [24.06892697687466]
  This paper presents SparseDVFS, a fine-grained, sparse-aware DVFS framework for energy-efficient edge inference.<n>Our key insight is that operator sparsity is a primary metric for hardware frequency modulation.<n>SparseDVFS achieves an average 78.17% energy efficiency gain over state-of-the-art solutions.
  arXiv  Detail & Related papers  (2026-03-23T12:29:30Z)
• TIDAL: Temporally Interleaved Diffusion and Action Loop for High-Frequency VLA Control [15.534182843429043]
  Large-scale Vision-Language-Action (VLA) models offer semantic generalization but suffer from high inference latency.<n>We propose TIDAL, a hierarchical framework that decouples semantic reasoning from high-frequency actuation.<n> TIDAL operates as a backbone-agnostic module for diffusion-basedVLAs, using a dual-frequency architecture.
  arXiv  Detail & Related papers  (2026-01-21T12:43:11Z)
• FADPNet: Frequency-Aware Dual-Path Network for Face Super-Resolution [70.61549422952193]
  Face super-resolution (FSR) under limited computational costs remains an open problem.<n>Existing approaches typically treat all facial pixels equally, resulting in suboptimal allocation of computational resources.<n>We propose FADPNet, a Frequency-Aware Dual-Path Network that decomposes facial features into low- and high-frequency components.
  arXiv  Detail & Related papers  (2025-06-17T02:33:42Z)
• F2Net: A Frequency-Fused Network for Ultra-High Resolution Remote Sensing Segmentation [10.67983913373955]
  F2Net is a frequency-aware framework that decomposes UHR images into high- and low-frequency components for specialized processing.<n>A Hybrid-Frequency Fusion module integrates these observations, guided by two novel objectives.<n>F2Net achieves state-of-the-art performance with mIoU of 80.22 and 83.39, respectively.
  arXiv  Detail & Related papers  (2025-06-09T15:09:49Z)
• SPECTRE: An FFT-Based Efficient Drop-In Replacement to Self-Attention for Long Contexts [2.200751835496112]
  Long-context transformers face significant efficiency challenges due to the quadratic cost of self-attention.<n>We introduce SPECTRE, a method that replaces each attention head with a fast real FFT.<n>We extend this efficiency to autoregressive generation through our Prefix-FFT cache and enhance local feature representation with an optional wavelet module.
  arXiv  Detail & Related papers  (2025-02-25T17:43:43Z)
• Frequency-Aware Deepfake Detection: Improving Generalizability through Frequency Space Learning [81.98675881423131]
  This research addresses the challenge of developing a universal deepfake detector that can effectively identify unseen deepfake images. Existing frequency-based paradigms have relied on frequency-level artifacts introduced during the up-sampling in GAN pipelines to detect forgeries. We introduce a novel frequency-aware approach called FreqNet, centered around frequency domain learning, specifically designed to enhance the generalizability of deepfake detectors.
  arXiv  Detail & Related papers  (2024-03-12T01:28:00Z)
• User Assignment and Resource Allocation for Hierarchical Federated Learning over Wireless Networks [20.09415156099031]
  Hierarchical FL (HFL) can reduce energy consumption and latency through effective resource allocation and appropriate user assignment. This article proposes a spectrum resource optimization algorithm (SROA) and a two-stage CPU algorithm (TSIA) for HFL. Experimental results demonstrate the superiority of the proposed HFL framework over existing studies in energy and latency reduction.
  arXiv  Detail & Related papers  (2023-09-17T12:10:39Z)
• Adaptive Frequency Filters As Efficient Global Token Mixers [100.27957692579892]
  We show that adaptive frequency filters can serve as efficient global token mixers. We take AFF token mixers as primary neural operators to build a lightweight neural network, dubbed AFFNet.
  arXiv  Detail & Related papers  (2023-07-26T07:42:28Z)
• Adaptive Federated Pruning in Hierarchical Wireless Networks [69.6417645730093]
  Federated Learning (FL) is a privacy-preserving distributed learning framework where a server aggregates models updated by multiple devices without accessing their private datasets. In this paper, we introduce model pruning for HFL in wireless networks to reduce the neural network scale. We show that our proposed HFL with model pruning achieves similar learning accuracy compared with the HFL without model pruning and reduces about 50 percent communication cost.
  arXiv  Detail & Related papers  (2023-05-15T22:04:49Z)
• Hierarchical Over-the-Air FedGradNorm [50.756991828015316]
  Multi-task learning (MTL) is a learning paradigm to learn multiple related tasks simultaneously with a single shared network. We propose hierarchical over-the-air (HOTA) PFL with a dynamic weighting strategy which we call HOTA-FedGradNorm.
  arXiv  Detail & Related papers  (2022-12-14T18:54:46Z)
• Bayesian Federated Learning over Wireless Networks [87.37301441859925]
  Federated learning is a privacy-preserving and distributed training method using heterogeneous data sets stored at local devices. This paper presents an efficient modified BFL algorithm called scalableBFL (SBFL)
  arXiv  Detail & Related papers  (2020-12-31T07:32:44Z)
• EdgeBERT: Sentence-Level Energy Optimizations for Latency-Aware Multi-Task NLP Inference [82.1584439276834]
  Transformer-based language models such as BERT provide significant accuracy improvement for a multitude of natural language processing (NLP) tasks. We present EdgeBERT, an in-depth algorithm- hardware co-design for latency-aware energy optimization for multi-task NLP.
  arXiv  Detail & Related papers  (2020-11-28T19:21:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.