Multi-Task Decision-Making for Multi-User 360 Video Processing over Wireless Networks
- URL: http://arxiv.org/abs/2407.03426v1
- Date: Wed, 3 Jul 2024 18:09:25 GMT
- Title: Multi-Task Decision-Making for Multi-User 360 Video Processing over Wireless Networks
- Authors: Babak Badnava, Jacob Chakareski, Morteza Hashemi,
- Abstract summary: We study a multi-task decision-making problem for 360 video processing in a wireless multi-user virtual reality (VR) system.
This comes at the expense of increased data volume and required bandwidth.
We propose a constrained quality of experience (QoE) problem in which the rebuffering time and quality variation between video frames are rely by user and video requirements.
- Score: 9.642593500545997
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We study a multi-task decision-making problem for 360 video processing in a wireless multi-user virtual reality (VR) system that includes an edge computing unit (ECU) to deliver 360 videos to VR users and offer computing assistance for decoding/rendering of video frames. However, this comes at the expense of increased data volume and required bandwidth. To balance this trade-off, we formulate a constrained quality of experience (QoE) maximization problem in which the rebuffering time and quality variation between video frames are bounded by user and video requirements. To solve the formulated multi-user QoE maximization, we leverage deep reinforcement learning (DRL) for multi-task rate adaptation and computation distribution (MTRC). The proposed MTRC approach does not rely on any predefined assumption about the environment and relies on video playback statistics (i.e., past throughput, decoding time, transmission time, etc.), video information, and the resulting performance to adjust the video bitrate and computation distribution. We train MTRC with real-world wireless network traces and 360 video datasets to obtain evaluation results in terms of the average QoE, peak signal-to-noise ratio (PSNR), rebuffering time, and quality variation. Our results indicate that the MTRC improves the users' QoE compared to state-of-the-art rate adaptation algorithm. Specifically, we show a 5.97 dB to 6.44 dB improvement in PSNR, a 1.66X to 4.23X improvement in rebuffering time, and a 4.21 dB to 4.35 dB improvement in quality variation.
Related papers
- MADRL-Based Rate Adaptation for 360° Video Streaming with Multi-Viewpoint Prediction [3.8611070161950916]
A key challenge of 360deg video playback is ensuring a high quality of experience (QoE) with limited network bandwidth.
Currently, most studies focus on tile-based adaptive (ABR) streaming based on single viewport prediction to reduce bandwidth consumption.
This paper first presents a multimodal spatial-temporal attention transformer to generate multiple viewpoint trajectories with their probabilities given a historical trajectory.
After that, a multi-agent deep reinforcement learning (MADRL)-based ABR algorithm utilizing multi-viewpoint prediction for 360deg video streaming is proposed.
arXiv Detail & Related papers (2024-05-13T13:59:59Z) - Video Compression with Arbitrary Rescaling Network [8.489428003916622]
We propose a rate-guided arbitrary rescaling network (RARN) for video resizing before encoding.
The lightweight RARN structure can process FHD (1080p) content at real-time speed (91 FPS) and obtain a considerable rate reduction.
arXiv Detail & Related papers (2023-06-07T07:15:18Z) - Towards High-Quality and Efficient Video Super-Resolution via
Spatial-Temporal Data Overfitting [27.302681897961588]
Deep convolutional neural networks (DNNs) are widely used in various fields of computer vision.
We propose a novel method for high-quality and efficient video resolution upscaling tasks.
We deploy our models on an off-the-shelf mobile phone, and experimental results show that our method achieves real-time video super-resolution with high video quality.
arXiv Detail & Related papers (2023-03-15T02:40:02Z) - Event-guided Multi-patch Network with Self-supervision for Non-uniform
Motion Deblurring [113.96237446327795]
We present a novel self-supervised event-guided deep hierarchical Multi-patch Network to deal with blurry images and videos.
We also propose an event-guided architecture to exploit motion cues contained in videos to tackle complex blur in videos.
Our MPN achieves the state of the art on the GoPro and VideoDeblurring datasets with a 40x faster runtime compared to current multi-scale methods.
arXiv Detail & Related papers (2023-02-14T15:58:00Z) - Distortion-Aware Loop Filtering of Intra 360^o Video Coding with
Equirectangular Projection [81.63407194858854]
We propose a distortion-aware loop filtering model to improve the performance of intra coding for 360$o$ videos projected via equirectangular projection (ERP) format.
Our proposed module analyzes content characteristics based on a coding unit (CU) partition mask and processes them through partial convolution to activate the specified area.
arXiv Detail & Related papers (2022-02-20T12:00:18Z) - Blind VQA on 360{\deg} Video via Progressively Learning from Pixels,
Frames and Video [66.57045901742922]
Blind visual quality assessment (BVQA) on 360textdegree video plays a key role in optimizing immersive multimedia systems.
In this paper, we take into account the progressive paradigm of human perception towards spherical video quality.
We propose a novel BVQA approach (namely ProVQA) for 360textdegree video via progressively learning from pixels, frames and video.
arXiv Detail & Related papers (2021-11-18T03:45:13Z) - Feeling of Presence Maximization: mmWave-Enabled Virtual Reality Meets
Deep Reinforcement Learning [76.46530937296066]
This paper investigates the problem of providing ultra-reliable and energy-efficient virtual reality (VR) experiences for wireless mobile users.
To ensure reliable ultra-high-definition (UHD) video frame delivery to mobile users, a coordinated multipoint (CoMP) transmission technique and millimeter wave (mmWave) communications are exploited.
arXiv Detail & Related papers (2021-06-03T08:35:10Z) - Cross Layer Optimization and Distributed Reinforcement Learning Approach
for Tile-Based 360 Degree Wireless Video Streaming [63.14489142588682]
We show that the problem can be decoupled into two interrelated subproblems.
We prove that the physical layer subproblem can be solved optimally with low complexity.
An actor-critic deep reinforcement learning (DRL) is proposed to leverage the parallel training of multiple independent agents.
arXiv Detail & Related papers (2020-11-12T12:59:10Z) - Real-world Video Adaptation with Reinforcement Learning [38.26695924173461]
Client-side video players employ adaptive (ABR) algorithms to optimize user quality of experience (QoE)
We evaluate recently proposed RL-based ABR methods in Facebook's web-based video streaming platform.
In a week-long worldwide deployment with more than 30 million video streaming sessions, our RL approach outperforms the existing human-engineered ABR algorithms.
arXiv Detail & Related papers (2020-08-28T21:44:24Z) - Non-Cooperative Game Theory Based Rate Adaptation for Dynamic Video
Streaming over HTTP [89.30855958779425]
Dynamic Adaptive Streaming over HTTP (DASH) has demonstrated to be an emerging and promising multimedia streaming technique.
We propose a novel algorithm to optimally allocate the limited export bandwidth of the server to multi-users to maximize their Quality of Experience (QoE) with fairness guaranteed.
arXiv Detail & Related papers (2019-12-27T01:19:14Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.