DeepWiVe: Deep-Learning-Aided Wireless Video Transmission

• URL: http://arxiv.org/abs/2111.13034v1
• Date: Thu, 25 Nov 2021 11:34:24 GMT
• Title: DeepWiVe: Deep-Learning-Aided Wireless Video Transmission
• Authors: Tze-Yang Tung and Deniz G\"und\"uz
• Abstract summary: We present DeepWiVe, the first-ever end-to-end joint source-channel coding (JSCC) video transmission scheme. We use deep neural networks (DNNs) to map video signals to channel symbols, combining video compression, channel coding, and modulation steps into a single neural transform. Our results show that DeepWiVe can overcome the cliff-effect, which is prevalent in conventional separation-based digital communication schemes.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We present DeepWiVe, the first-ever end-to-end joint source-channel coding (JSCC) video transmission scheme that leverages the power of deep neural networks (DNNs) to directly map video signals to channel symbols, combining video compression, channel coding, and modulation steps into a single neural transform. Our DNN decoder predicts residuals without distortion feedback, which improves video quality by accounting for occlusion/disocclusion and camera movements. We simultaneously train different bandwidth allocation networks for the frames to allow variable bandwidth transmission. Then, we train a bandwidth allocation network using reinforcement learning (RL) that optimizes the allocation of limited available channel bandwidth among video frames to maximize overall visual quality. Our results show that DeepWiVe can overcome the cliff-effect, which is prevalent in conventional separation-based digital communication schemes, and achieve graceful degradation with the mismatch between the estimated and actual channel qualities. DeepWiVe outperforms H.264 video compression followed by low-density parity check (LDPC) codes in all channel conditions by up to 0.0462 on average in terms of the multi-scale structural similarity index measure (MS-SSIM), while beating H.265 + LDPC by up to 0.0058 on average. We also illustrate the importance of optimizing bandwidth allocation in JSCC video transmission by showing that our optimal bandwidth allocation policy is superior to the na\"ive uniform allocation. We believe this is an important step towards fulfilling the potential of an end-to-end optimized JSCC wireless video transmission system that is superior to the current separation-based designs.

Related papers

• VideoQA-SC: Adaptive Semantic Communication for Video Question Answering [21.0279034601774]
  We propose an end-to-end SC system for video question answering tasks called VideoQA-SC. Our goal is to accomplish VideoQA tasks directly based on video semantics over noisy or fading wireless channels. Our results show the great potential of task-oriented SC system design for video applications.
  arXiv  Detail & Related papers  (2024-05-17T06:11:10Z)
• Sandwiched Video Compression: Efficiently Extending the Reach of Standard Codecs with Neural Wrappers [11.968545394054816]
  We propose a video compression system that wraps neural networks around a standard video. Networks are trained jointly to optimize a rate-distortion loss function. We observe 30% improvements in rate at the same quality over HEVC.
  arXiv  Detail & Related papers  (2023-03-20T22:03:44Z)
• Graph Neural Networks for Channel Decoding [71.15576353630667]
  We showcase competitive decoding performance for various coding schemes, such as low-density parity-check (LDPC) and BCH codes. The idea is to let a neural network (NN) learn a generalized message passing algorithm over a given graph. We benchmark our proposed decoder against state-of-the-art in conventional channel decoding as well as against recent deep learning-based results.
  arXiv  Detail & Related papers  (2022-07-29T15:29:18Z)
• Wireless Deep Video Semantic Transmission [14.071114007641313]
  We propose a new class of high-efficiency deep joint source-channel coding methods to achieve end-to-end video transmission over wireless channels. Our framework is collected under the name deep video semantic transmission (DVST)
  arXiv  Detail & Related papers  (2022-05-26T03:26:43Z)
• A Coding Framework and Benchmark towards Low-Bitrate Video Understanding [63.05385140193666]
  We propose a traditional-neural mixed coding framework that takes advantage of both traditional codecs and neural networks (NNs) The framework is optimized by ensuring that a transportation-efficient semantic representation of the video is preserved. We build a low-bitrate video understanding benchmark with three downstream tasks on eight datasets, demonstrating the notable superiority of our approach.
  arXiv  Detail & Related papers  (2022-02-06T16:29:15Z)
• IoV Scenario: Implementation of a Bandwidth Aware Algorithm in Wireless Network Communication Mode [49.734868032441625]
  This paper proposes a bandwidth aware multi domain virtual network embedding algorithm (BA-VNE) The algorithm is mainly aimed at the problem that users need a lot of bandwidth in wireless communication mode. In order to improve the performance of the algorithm, we introduce particle swarm optimization (PSO) algorithm.
  arXiv  Detail & Related papers  (2022-02-03T03:34:06Z)
• An Adaptive Device-Edge Co-Inference Framework Based on Soft Actor-Critic [72.35307086274912]
  High-dimension parameter model and large-scale mathematical calculation restrict execution efficiency, especially for Internet of Things (IoT) devices. We propose a new Deep Reinforcement Learning (DRL)-Soft Actor Critic for discrete (SAC-d), which generates the emphexit point, emphexit point, and emphcompressing bits by soft policy iterations. Based on the latency and accuracy aware reward design, such an computation can well adapt to the complex environment like dynamic wireless channel and arbitrary processing, and is capable of supporting the 5G URL
  arXiv  Detail & Related papers  (2022-01-09T09:31:50Z)
• Optical-Flow-Reuse-Based Bidirectional Recurrent Network for Space-Time Video Super-Resolution [52.899234731501075]
  Space-time video super-resolution (ST-VSR) simultaneously increases the spatial resolution and frame rate for a given video. Existing methods typically suffer from difficulties in how to efficiently leverage information from a large range of neighboring frames. We propose a coarse-to-fine bidirectional recurrent neural network instead of using ConvLSTM to leverage knowledge between adjacent frames.
  arXiv  Detail & Related papers  (2021-10-13T15:21:30Z)
• Multi-Density Attention Network for Loop Filtering in Video Compression [9.322800480045336]
  We propose a on-line scaling based multi-density attention network for loop filtering in video compression. Experimental results show that 10.18% bit-rate reduction at the same video quality can be achieved over the latest Versatile Video Coding (VVC) standard.
  arXiv  Detail & Related papers  (2021-04-08T05:46:38Z)
• Deep Learning-based Resource Allocation For Device-to-Device Communication [66.74874646973593]
  We propose a framework for the optimization of the resource allocation in multi-channel cellular systems with device-to-device (D2D) communication. A deep learning (DL) framework is proposed, where the optimal resource allocation strategy for arbitrary channel conditions is approximated by deep neural network (DNN) models. Our simulation results confirm that near-optimal performance can be attained with low time, which underlines the real-time capability of the proposed scheme.
  arXiv  Detail & Related papers  (2020-11-25T14:19:23Z)
• NANCY: Neural Adaptive Network Coding methodologY for video distribution over wireless networks [1.636104578028594]
  NANCY is a system that generates adaptive bit rates (ABR) for video and adaptive network coding rates (ANCR) NANCY trains a neural network model with rewards formulated as quality of experience (QoE) metrics. Our results show that NANCY provides 29.91% and 60.34% higher average QoE than Pensieve and robustMPC, respectively.
  arXiv  Detail & Related papers  (2020-08-21T15:55:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.