DeePref: Deep Reinforcement Learning For Video Prefetching In Content Delivery Networks

• URL: http://arxiv.org/abs/2310.07881v1
• Date: Wed, 11 Oct 2023 20:45:46 GMT
• Title: DeePref: Deep Reinforcement Learning For Video Prefetching In Content Delivery Networks
• Authors: Nawras Alkassab, Chin-Tser Huang, Tania Lorido Botran
• Abstract summary: We propose DeePref, a Deep Reinforcement Learning agent for online video content prefetching in Content Delivery Networks. Our results show that DeePref DRQN, using a real-world dataset, achieves a 17% increase in prefetching accuracy and a 28% increase in prefetching coverage on average.
• Score: 0.06138671548064355
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Content Delivery Networks carry the majority of Internet traffic, and the increasing demand for video content as a major IP traffic across the Internet highlights the importance of caching and prefetching optimization algorithms. Prefetching aims to make data available in the cache before the requester places its request to reduce access time and improve the Quality of Experience on the user side. Prefetching is well investigated in operating systems, compiler instructions, in-memory cache, local storage systems, high-speed networks, and cloud systems. Traditional prefetching techniques are well adapted to a particular access pattern, but fail to adapt to sudden variations or randomization in workloads. This paper explores the use of reinforcement learning to tackle the changes in user access patterns and automatically adapt over time. To this end, we propose, DeePref, a Deep Reinforcement Learning agent for online video content prefetching in Content Delivery Networks. DeePref is a prefetcher implemented on edge networks and is agnostic to hardware design, operating systems, and applications. Our results show that DeePref DRQN, using a real-world dataset, achieves a 17% increase in prefetching accuracy and a 28% increase in prefetching coverage on average compared to baseline approaches that use video content popularity as a building block to statically or dynamically make prefetching decisions. We also study the possibility of transfer learning of statistical models from one edge network into another, where unseen user requests from unknown distribution are observed. In terms of transfer learning, the increase in prefetching accuracy and prefetching coverage are [$30%$, $10%$], respectively. Our source code will be available on Github.

Related papers

• Resource-Aware Hierarchical Federated Learning in Wireless Video Caching Networks [24.664469755746463]
  Backhaul traffic congestion caused by the video traffic of a few popular files can be alleviated by storing the to-be-requested content. We propose a novel resource-aware hierarchical federated learning (RawHFL) solution for predicting user's future content requests.
  arXiv  Detail & Related papers  (2024-02-06T18:17:02Z)
• Resource-Aware Hierarchical Federated Learning for Video Caching in Wireless Networks [29.137803674759848]
  A privacy-preserving method is desirable to learn how users' demands change over time. This paper proposes a novel resource-aware hierarchical federated learning (RawHFL) solution to predict users' future content requests. Our simulation results show that the proposed solution significantly outperforms the considered baselines in terms of prediction accuracy and total energy expenditure.
  arXiv  Detail & Related papers  (2023-11-12T18:23:17Z)
• Reinforcement Learning -based Adaptation and Scheduling Methods for Multi-source DASH [1.1971219484941955]
  Dynamic adaptive streaming over HTTP (DASH) has been widely used in video streaming recently. In multi-source streaming, video chunks may arrive out of order due to different conditions of the network paths. This paper proposes two algorithms for streaming from multiple sources: RL-based adaptation with greedy scheduling (RLAGS) and RL-based adaptation and scheduling (RLAS)
  arXiv  Detail & Related papers  (2023-07-25T06:47:12Z)
• SpawnNet: Learning Generalizable Visuomotor Skills from Pre-trained Networks [52.766795949716986]
  We present a study of the generalization capabilities of the pre-trained visual representations at the categorical level. We propose SpawnNet, a novel two-stream architecture that learns to fuse pre-trained multi-layer representations into a separate network to learn a robust policy.
  arXiv  Detail & Related papers  (2023-07-07T13:01:29Z)
• Distributed Reinforcement Learning for Privacy-Preserving Dynamic Edge Caching [91.50631418179331]
  A privacy-preserving distributed deep policy gradient (P2D3PG) is proposed to maximize the cache hit rates of devices in the MEC networks. We convert the distributed optimizations into model-free Markov decision process problems and then introduce a privacy-preserving federated learning method for popularity prediction.
  arXiv  Detail & Related papers  (2021-10-20T02:48:27Z)
• Semi-supervised Network Embedding with Differentiable Deep Quantisation [81.49184987430333]
  We develop d-SNEQ, a differentiable quantisation method for network embedding. d-SNEQ incorporates a rank loss to equip the learned quantisation codes with rich high-order information. It is able to substantially compress the size of trained embeddings, thus reducing storage footprint and accelerating retrieval speed.
  arXiv  Detail & Related papers  (2021-08-20T11:53:05Z)
• A Survey of Deep Learning for Data Caching in Edge Network [1.9798034349981157]
  This paper summarizes the utilization of deep learning for data caching in edge network. We first outline the typical research topics in content caching and formulate a taxonomy based on network hierarchical structure. Then, a number of key types of deep learning algorithms are presented, ranging from supervised learning to unsupervised learning as well as reinforcement learning.
  arXiv  Detail & Related papers  (2020-08-17T12:02:32Z)
• Caching Placement and Resource Allocation for Cache-Enabling UAV NOMA Networks [87.6031308969681]
  This article investigates the cache-enabling unmanned aerial vehicle (UAV) cellular networks with massive access capability supported by non-orthogonal multiple access (NOMA) We formulate the long-term caching placement and resource allocation optimization problem for content delivery delay minimization as a Markov decision process (MDP) We propose a Q-learning based caching placement and resource allocation algorithm, where the UAV learns and selects action with emphsoft $varepsilon$-greedy strategy to search for the optimal match between actions and states.
  arXiv  Detail & Related papers  (2020-08-12T08:33:51Z)
• Reinforcement Learning for Caching with Space-Time Popularity Dynamics [61.55827760294755]
  caching is envisioned to play a critical role in next-generation networks. To intelligently prefetch and store contents, a cache node should be able to learn what and when to cache. This chapter presents a versatile reinforcement learning based approach for near-optimal caching policy design.
  arXiv  Detail & Related papers  (2020-05-19T01:23:51Z)
• Accelerating Deep Reinforcement Learning With the Aid of Partial Model: Energy-Efficient Predictive Video Streaming [97.75330397207742]
  Predictive power allocation is conceived for energy-efficient video streaming over mobile networks using deep reinforcement learning. To handle the continuous state and action spaces, we resort to deep deterministic policy gradient (DDPG) algorithm. Our simulation results show that the proposed policies converge to the optimal policy that is derived based on perfect large-scale channel prediction.
  arXiv  Detail & Related papers  (2020-03-21T17:36:53Z)
• PA-Cache: Evolving Learning-Based Popularity-Aware Content Caching in Edge Networks [14.939950326112045]
  We propose an evolving learning-based content caching policy, named PA-Cache in edge networks. It adaptively learns time-varying content popularity and determines which contents should be replaced when the cache is full. We extensively evaluate the performance of our proposed PA-Cache on real-world traces from a large online video-on-demand service provider.
  arXiv  Detail & Related papers  (2020-02-20T15:38:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.