Multi-Task Decision-Making for Multi-User 360 Video Processing over Wireless Networks

• URL: http://arxiv.org/abs/2407.03426v1
• Date: Wed, 3 Jul 2024 18:09:25 GMT
• Title: Multi-Task Decision-Making for Multi-User 360 Video Processing over Wireless Networks
• Authors: Babak Badnava, Jacob Chakareski, Morteza Hashemi,
• Abstract summary: We study a multi-task decision-making problem for 360 video processing in a wireless multi-user virtual reality (VR) system. This comes at the expense of increased data volume and required bandwidth. We propose a constrained quality of experience (QoE) problem in which the rebuffering time and quality variation between video frames are rely by user and video requirements.
• Score: 9.642593500545997
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We study a multi-task decision-making problem for 360 video processing in a wireless multi-user virtual reality (VR) system that includes an edge computing unit (ECU) to deliver 360 videos to VR users and offer computing assistance for decoding/rendering of video frames. However, this comes at the expense of increased data volume and required bandwidth. To balance this trade-off, we formulate a constrained quality of experience (QoE) maximization problem in which the rebuffering time and quality variation between video frames are bounded by user and video requirements. To solve the formulated multi-user QoE maximization, we leverage deep reinforcement learning (DRL) for multi-task rate adaptation and computation distribution (MTRC). The proposed MTRC approach does not rely on any predefined assumption about the environment and relies on video playback statistics (i.e., past throughput, decoding time, transmission time, etc.), video information, and the resulting performance to adjust the video bitrate and computation distribution. We train MTRC with real-world wireless network traces and 360 video datasets to obtain evaluation results in terms of the average QoE, peak signal-to-noise ratio (PSNR), rebuffering time, and quality variation. Our results indicate that the MTRC improves the users' QoE compared to state-of-the-art rate adaptation algorithm. Specifically, we show a 5.97 dB to 6.44 dB improvement in PSNR, a 1.66X to 4.23X improvement in rebuffering time, and a 4.21 dB to 4.35 dB improvement in quality variation.

Related papers

• EPS: Efficient Patch Sampling for Video Overfitting in Deep Super-Resolution Model Training [15.684865589513597]
  We propose an efficient patch sampling method named EPS for video SR network overfitting. Our method reduces the number of patches for the training to 4% to 25%, depending on the resolution and number of clusters. Compared to the state-of-the-art patch sampling method, EMT, our approach achieves an 83% decrease in overall run time.
  arXiv  Detail & Related papers  (2024-11-25T12:01:57Z)
• LMM-VQA: Advancing Video Quality Assessment with Large Multimodal Models [53.64461404882853]
  Video quality assessment (VQA) algorithms are needed to monitor and optimize the quality of streaming videos. Here, we propose the first Large Multi-Modal Video Quality Assessment (LMM-VQA) model, which introduces a novel visual modeling strategy for quality-aware feature extraction.
  arXiv  Detail & Related papers  (2024-08-26T04:29:52Z)
• MADRL-Based Rate Adaptation for 360° Video Streaming with Multi-Viewpoint Prediction [3.8611070161950916]
  A key challenge of 360deg video playback is ensuring a high quality of experience (QoE) with limited network bandwidth. Currently, most studies focus on tile-based adaptive (ABR) streaming based on single viewport prediction to reduce bandwidth consumption. This paper first presents a multimodal spatial-temporal attention transformer to generate multiple viewpoint trajectories with their probabilities given a historical trajectory. After that, a multi-agent deep reinforcement learning (MADRL)-based ABR algorithm utilizing multi-viewpoint prediction for 360deg video streaming is proposed.
  arXiv  Detail & Related papers  (2024-05-13T13:59:59Z)
• Video Compression with Arbitrary Rescaling Network [8.489428003916622]
  We propose a rate-guided arbitrary rescaling network (RARN) for video resizing before encoding. The lightweight RARN structure can process FHD (1080p) content at real-time speed (91 FPS) and obtain a considerable rate reduction.
  arXiv  Detail & Related papers  (2023-06-07T07:15:18Z)
• Distortion-Aware Loop Filtering of Intra 360^o Video Coding with Equirectangular Projection [81.63407194858854]
  We propose a distortion-aware loop filtering model to improve the performance of intra coding for 360$o$ videos projected via equirectangular projection (ERP) format. Our proposed module analyzes content characteristics based on a coding unit (CU) partition mask and processes them through partial convolution to activate the specified area.
  arXiv  Detail & Related papers  (2022-02-20T12:00:18Z)
• Blind VQA on 360{\deg} Video via Progressively Learning from Pixels, Frames and Video [66.57045901742922]
  Blind visual quality assessment (BVQA) on 360textdegree video plays a key role in optimizing immersive multimedia systems. In this paper, we take into account the progressive paradigm of human perception towards spherical video quality. We propose a novel BVQA approach (namely ProVQA) for 360textdegree video via progressively learning from pixels, frames and video.
  arXiv  Detail & Related papers  (2021-11-18T03:45:13Z)
• Feeling of Presence Maximization: mmWave-Enabled Virtual Reality Meets Deep Reinforcement Learning [76.46530937296066]
  This paper investigates the problem of providing ultra-reliable and energy-efficient virtual reality (VR) experiences for wireless mobile users. To ensure reliable ultra-high-definition (UHD) video frame delivery to mobile users, a coordinated multipoint (CoMP) transmission technique and millimeter wave (mmWave) communications are exploited.
  arXiv  Detail & Related papers  (2021-06-03T08:35:10Z)
• Cross Layer Optimization and Distributed Reinforcement Learning for Wireless 360° Video Streaming [54.60967639512643]
  We propose a cross layer optimization approach that maximizes the available rate to each user and efficiently uses it to maximize users' QoE. We show that the problem can be decoupled into two interrelated subproblems. An actor-critic deep reinforcement learning (DRL) is proposed to leverage the parallel training of multiple independent agents and solve the application layer subproblem.
  arXiv  Detail & Related papers  (2020-11-12T12:59:10Z)
• Real-world Video Adaptation with Reinforcement Learning [38.26695924173461]
  Client-side video players employ adaptive (ABR) algorithms to optimize user quality of experience (QoE) We evaluate recently proposed RL-based ABR methods in Facebook's web-based video streaming platform. In a week-long worldwide deployment with more than 30 million video streaming sessions, our RL approach outperforms the existing human-engineered ABR algorithms.
  arXiv  Detail & Related papers  (2020-08-28T21:44:24Z)
• Non-Cooperative Game Theory Based Rate Adaptation for Dynamic Video Streaming over HTTP [89.30855958779425]
  Dynamic Adaptive Streaming over HTTP (DASH) has demonstrated to be an emerging and promising multimedia streaming technique. We propose a novel algorithm to optimally allocate the limited export bandwidth of the server to multi-users to maximize their Quality of Experience (QoE) with fairness guaranteed.
  arXiv  Detail & Related papers  (2019-12-27T01:19:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.