Edge Caching Based on Deep Reinforcement Learning and Transfer Learning
- URL: http://arxiv.org/abs/2402.14576v2
- Date: Fri, 1 Mar 2024 00:21:38 GMT
- Title: Edge Caching Based on Deep Reinforcement Learning and Transfer Learning
- Authors: Farnaz Niknia, Ping Wang, Zixu Wang, Aakash Agarwal and Adib S. Rezaei
- Abstract summary: Surge in traffic has strained backhaul links and backbone networks, prompting the exploration of caching solutions at the edge router.
We formulate the caching problem using a semi-Markov Decision Process (SMDP) to accommodate the continuous-time nature of real-world scenarios.
We propose a double deep Q-learning-based caching approach that comprehensively accounts for file features such as lifetime, size, and importance.
- Score: 4.568097048023971
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: This paper addresses the escalating challenge of redundant data transmission
in networks. The surge in traffic has strained backhaul links and backbone
networks, prompting the exploration of caching solutions at the edge router.
Existing work primarily relies on Markov Decision Processes (MDP) for caching
issues, assuming fixed-time interval decisions; however, real-world scenarios
involve random request arrivals, and despite the critical role of various file
characteristics in determining an optimal caching policy, none of the related
existing work considers all these file characteristics in forming a caching
policy. In this paper, first, we formulate the caching problem using a
semi-Markov Decision Process (SMDP) to accommodate the continuous-time nature
of real-world scenarios allowing for caching decisions at random times upon
file requests. Then, we propose a double deep Q-learning-based caching approach
that comprehensively accounts for file features such as lifetime, size, and
importance. Simulation results demonstrate the superior performance of our
approach compared to a recent Deep Reinforcement Learning-based method.
Furthermore, we extend our work to include a Transfer Learning (TL) approach to
account for changes in file request rates in the SMDP framework. The proposed
TL approach exhibits fast convergence, even in scenarios with increased
differences in request rates between source and target domains, presenting a
promising solution to the dynamic challenges of caching in real-world
environments.
Related papers
- Efficient Inference of Vision Instruction-Following Models with Elastic Cache [76.44955111634545]
We introduce Elastic Cache, a novel strategy for efficient deployment of instruction-following large vision-language models.
We propose an importance-driven cache merging strategy to prune redundancy caches.
For instruction encoding, we utilize the frequency to evaluate the importance of caches.
Results on a range of LVLMs demonstrate that Elastic Cache not only boosts efficiency but also notably outperforms existing pruning methods in language generation.
arXiv Detail & Related papers (2024-07-25T15:29:05Z) - A Learning-Based Caching Mechanism for Edge Content Delivery [2.412158290827225]
5G networks and the rise of the Internet of Things (IoT) are increasingly extending into the network edge.
This shift introduces unique challenges, particularly due to the limited cache storage and the diverse request patterns at the edge.
We introduce HR-Cache, a learning-based caching framework grounded in the principles of Hazard Rate (HR) ordering.
arXiv Detail & Related papers (2024-02-05T08:06:03Z) - Optimistic No-regret Algorithms for Discrete Caching [6.182368229968862]
We take a systematic look at the problem of storing whole files in a cache with limited capacity in the context of optimistic learning.
We provide a universal lower bound for prediction-assisted online caching and design a suite of policies with a range of performance-complexity trade-offs.
Our results substantially improve upon all recently-proposed online caching policies, which, being unable to exploit the oracle predictions, offer only $O(sqrtT)$ regret.
arXiv Detail & Related papers (2022-08-15T09:18:41Z) - Accelerating Deep Learning Classification with Error-controlled
Approximate-key Caching [72.50506500576746]
We propose a novel caching paradigm, that we named approximate-key caching.
While approximate cache hits alleviate DL inference workload and increase the system throughput, they however introduce an approximation error.
We analytically model our caching system performance for classic LRU and ideal caches, we perform a trace-driven evaluation of the expected performance, and we compare the benefits of our proposed approach with the state-of-the-art similarity caching.
arXiv Detail & Related papers (2021-12-13T13:49:11Z) - Learning from Images: Proactive Caching with Parallel Convolutional
Neural Networks [94.85780721466816]
A novel framework for proactive caching is proposed in this paper.
It combines model-based optimization with data-driven techniques by transforming an optimization problem into a grayscale image.
Numerical results show that the proposed scheme can reduce 71.6% computation time with only 0.8% additional performance cost.
arXiv Detail & Related papers (2021-08-15T21:32:47Z) - Cocktail Edge Caching: Ride Dynamic Trends of Content Popularity with
Ensemble Learning [10.930268276150262]
Edge caching will play a critical role in facilitating the emerging content-rich applications.
It faces many new challenges, in particular, the highly dynamic content popularity and the heterogeneous caching computation.
We propose Cocktail Edge Caching, that tackles the dynamic popularity and heterogeneity through ensemble learning.
arXiv Detail & Related papers (2021-01-14T21:59:04Z) - Caching Placement and Resource Allocation for Cache-Enabling UAV NOMA
Networks [87.6031308969681]
This article investigates the cache-enabling unmanned aerial vehicle (UAV) cellular networks with massive access capability supported by non-orthogonal multiple access (NOMA)
We formulate the long-term caching placement and resource allocation optimization problem for content delivery delay minimization as a Markov decision process (MDP)
We propose a Q-learning based caching placement and resource allocation algorithm, where the UAV learns and selects action with emphsoft $varepsilon$-greedy strategy to search for the optimal match between actions and states.
arXiv Detail & Related papers (2020-08-12T08:33:51Z) - Reinforcement Learning for Caching with Space-Time Popularity Dynamics [61.55827760294755]
caching is envisioned to play a critical role in next-generation networks.
To intelligently prefetch and store contents, a cache node should be able to learn what and when to cache.
This chapter presents a versatile reinforcement learning based approach for near-optimal caching policy design.
arXiv Detail & Related papers (2020-05-19T01:23:51Z) - A Non-Stationary Bandit-Learning Approach to Energy-Efficient
Femto-Caching with Rateless-Coded Transmission [98.47527781626161]
We study a resource allocation problem for joint caching and transmission in small cell networks.
We then formulate the problem as to select a file from the cache together with a transmission power level for every broadcast round.
In contrast to the state-of-the-art research, the proposed approach is especially suitable for networks with time-variant statistical properties.
arXiv Detail & Related papers (2020-04-13T09:07:17Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.