Exploiting Deep Reinforcement Learning for Edge Caching in Cell-Free Massive MIMO Systems

• URL: http://arxiv.org/abs/2208.12453v2
• Date: Wed, 11 Sep 2024 02:33:06 GMT
• Title: Exploiting Deep Reinforcement Learning for Edge Caching in Cell-Free Massive MIMO Systems
• Abstract summary: Cell-free massive multiple-input-multiple-output is promising to meet the stringent quality-of-experience (QoE) requirements of railway wireless communications. Key challenge is how to deliver the desired contents timely due to the radical changing propagation environment caused by the growing train speed. We propose to proactively cache the likely-requesting contents at the upcoming APs which perform the coherent transmission to reduce end-to-end delay.
• Score: 17.901600491149175
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Cell-free massive multiple-input-multiple-output is promising to meet the stringent quality-of-experience (QoE) requirements of railway wireless communications by coordinating many successional access points (APs) to serve the onboard users coherently. A key challenge is how to deliver the desired contents timely due to the radical changing propagation environment caused by the growing train speed. In this paper, we propose to proactively cache the likely-requesting contents at the upcoming APs which perform the coherent transmission to reduce end-to-end delay. A long-term QoE-maximization problem is formulated and two cache placement algorithms are proposed. One is based on heuristic convex optimization (HCO) and the other exploits deep reinforcement learning (DRL) with soft actor-critic (SAC). Compared to the conventional benchmark, numerical results show the advantage of our proposed algorithms on QoE and hit probability. With the advanced DRL model, SAC outperforms HCO on QoE by predicting the user requests accurately.

Related papers

• ECHO: Early-layer Collaborative Hierarchical Orchestration with Bonus Logits in Speculative Decoding [60.34475274870224]
  ECHO is a hierarchical dual-loop framework that exploits the functional asymmetry between LLM layers.<n>Within the inner loop, early-layer bonus logits drive rapid, multi-step draft-tree exploration at a minimal cost.<n>ECHO significantly boosts mean accepted tokens and achieves a 2.4$times$ to 2.9$times$ speedup.
  arXiv  Detail & Related papers  (2026-09-15T14:21:33Z)
• Hardware-Efficient Quantum Optimization for Transportation Networks via Compressed Adiabatic Evolution [1.409283414986451]
  Transportation systems such as urban logistics, vehicle routing, and infrastructure planning require solving large-scale optimization problems under complex constraints.<n>We develop a hardware-grounded hybrid quantum optimization framework that uses Approximate Quantum Compilation (AQC) to compress early segments of digitized adiabatic evolution into shallow circuits.<n>Results show that moderate prefix compression reduces two-qubit gate depth while maintaining or improving feasible solution, particularly for routing problems.
  arXiv  Detail & Related papers  (2026-04-28T23:45:54Z)
• KnapSpec: Self-Speculative Decoding via Adaptive Layer Selection as a Knapsack Problem [12.668341559890605]
  We propose KnapSpec, a training-free framework that reformulates draft model selection as a knapsack problem to maximize tokens-per-time throughput.<n>We provide the first rigorous theoretical analysis establishing cosine similarity between hidden states as a mathematically sound proxy for the token acceptance rate.<n>Our experiments on Qwen3 and Llama3 demonstrate that KnapSpec consistently outperforms state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-02-23T08:13:03Z)
• Intelligent Optimization of Wireless Access Point Deployment for Communication-Based Train Control Systems Using Deep Reinforcement Learning [12.256904916760796]
  Urban railway systems increasingly rely on communication based train control (CBTC) systems.<n> optimal deployment of access points (APs) in tunnels is critical for robust wireless coverage.<n>Traditional methods, such as empirical model-based optimization algorithms, are hindered by excessive measurement requirements.
  arXiv  Detail & Related papers  (2025-09-29T14:07:44Z)
• MiCo: End-to-End Mixed Precision Neural Network Co-Exploration Framework for Edge AI [0.9208007322096533]
  Quantized Neural Networks (QNN) with extremely low-bitwidth data have proven promising in efficient storage and computation on edge devices.<n>To further reduce the accuracy drop while increasing speedup, layer-wise mixed-precision quantization (MPQ) becomes a popular solution.<n>We propose the MiCo framework, a holistic MPQ exploration and deployment framework for edge AI applications.
  arXiv  Detail & Related papers  (2025-08-13T05:18:21Z)
• Causal-Aware Intelligent QoE Optimization for VR Interaction with Adaptive Keyframe Extraction [6.2819120598047]
  This paper proposes an intelligent framework to maximize quality of experience (QoE) in VR games.<n>It integrates adaptive extraction with causal-aware reinforcement learning (RL)<n> Experiments show that our framework significantly reduces interactive latency, enhances QoE, and maintains fairness.
  arXiv  Detail & Related papers  (2025-06-24T07:32:34Z)
• Task-Oriented Feature Compression for Multimodal Understanding via Device-Edge Co-Inference [49.77734021302196]
  We propose a task-oriented feature compression (TOFC) method for multimodal understanding in a device-edge co-inference framework. To enhance compression efficiency, multiple entropy models are adaptively selected based on the characteristics of the visual features. Results show that TOFC achieves up to 60% reduction in data transmission overhead and 50% reduction in system latency.
  arXiv  Detail & Related papers  (2025-03-17T08:37:22Z)
• DEER: A Delay-Resilient Framework for Reinforcement Learning with Variable Delays [26.032139258562708]
  We propose $textbfDEER (Delay-resilient-Enhanced RL)$, a framework designed to effectively enhance the interpretability and address the random delay issues. In a variety of delayed scenarios, the trained encoder can seamlessly integrate with standard RL algorithms without requiring additional modifications. The results confirm that DEER is superior to state-of-the-art RL algorithms in both constant and random delay settings.
  arXiv  Detail & Related papers  (2024-06-05T09:45:26Z)
• Adaptive User-Centric Entanglement Routing in Quantum Data Networks [5.421492821020181]
  Distributed quantum computing (DQC) holds immense promise in harnessing the potential of quantum computing by interconnecting multiple small quantum computers (QCs) through a quantum data network (QDN) establishing long-distance quantum entanglement between two QCs for quantum teleportation within the QDN is a critical aspect, and it involves entanglement routing. Existing approaches have mainly focused on optimizing entanglement performance for current entanglement connection (EC) requests. We present a novel user-centric entanglement routing problem that spans an extended period to maximize entanglement success rate while adhering to the user's budget constraint.
  arXiv  Detail & Related papers  (2024-04-13T17:20:00Z)
• Learning RL-Policies for Joint Beamforming Without Exploration: A Batch Constrained Off-Policy Approach [1.0080317855851213]
  We consider the problem of network parameter cancellation optimization for networks. We show that deploying an algorithm in the real world for exploration and learning can be achieved with the data without exploring.
  arXiv  Detail & Related papers  (2023-10-12T18:36:36Z)
• Differentially Private Deep Q-Learning for Pattern Privacy Preservation in MEC Offloading [76.0572817182483]
  attackers may eavesdrop on the offloading decisions to infer the edge server's (ES's) queue information and users' usage patterns. We propose an offloading strategy which jointly minimizes the latency, ES's energy consumption, and task dropping rate, while preserving pattern privacy (PP) We develop a Differential Privacy Deep Q-learning based Offloading (DP-DQO) algorithm to solve this problem while addressing the PP issue by injecting noise into the generated offloading decisions.
  arXiv  Detail & Related papers  (2023-02-09T12:50:18Z)
• Collaborative Intelligent Reflecting Surface Networks with Multi-Agent Reinforcement Learning [63.83425382922157]
  Intelligent reflecting surface (IRS) is envisioned to be widely applied in future wireless networks. In this paper, we investigate a multi-user communication system assisted by cooperative IRS devices with the capability of energy harvesting.
  arXiv  Detail & Related papers  (2022-03-26T20:37:14Z)
• Real-Time GPU-Accelerated Machine Learning Based Multiuser Detection for 5G and Beyond [70.81551587109833]
  nonlinear beamforming filters can significantly outperform linear approaches in stationary scenarios with massive connectivity. One of the main challenges comes from the real-time implementation of these algorithms. This paper explores the acceleration of APSM-based algorithms through massive parallelization.
  arXiv  Detail & Related papers  (2022-01-13T15:20:45Z)
• An Adaptive Device-Edge Co-Inference Framework Based on Soft Actor-Critic [72.35307086274912]
  High-dimension parameter model and large-scale mathematical calculation restrict execution efficiency, especially for Internet of Things (IoT) devices. We propose a new Deep Reinforcement Learning (DRL)-Soft Actor Critic for discrete (SAC-d), which generates the emphexit point, emphexit point, and emphcompressing bits by soft policy iterations. Based on the latency and accuracy aware reward design, such an computation can well adapt to the complex environment like dynamic wireless channel and arbitrary processing, and is capable of supporting the 5G URL
  arXiv  Detail & Related papers  (2022-01-09T09:31:50Z)
• APQ: Joint Search for Network Architecture, Pruning and Quantization Policy [49.3037538647714]
  We present APQ for efficient deep learning inference on resource-constrained hardware. Unlike previous methods that separately search the neural architecture, pruning policy, and quantization policy, we optimize them in a joint manner. With the same accuracy, APQ reduces the latency/energy by 2x/1.3x over MobileNetV2+HAQ.
  arXiv  Detail & Related papers  (2020-06-15T16:09:17Z)
• Reinforcement Learning Based Cooperative Coded Caching under Dynamic Popularities in Ultra-Dense Networks [38.44125997148742]
  caching strategy at small base stations (SBSs) is critical to meet massive high data rate requests. We exploit reinforcement learning (RL) to design a cooperative caching strategy with maximum-distance separable (MDS) coding.
  arXiv  Detail & Related papers  (2020-03-08T10:45:45Z)
• Non-Cooperative Game Theory Based Rate Adaptation for Dynamic Video Streaming over HTTP [89.30855958779425]
  Dynamic Adaptive Streaming over HTTP (DASH) has demonstrated to be an emerging and promising multimedia streaming technique. We propose a novel algorithm to optimally allocate the limited export bandwidth of the server to multi-users to maximize their Quality of Experience (QoE) with fairness guaranteed.
  arXiv  Detail & Related papers  (2019-12-27T01:19:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.