Deep Neural Network-based Enhancement for Image and Video Streaming Systems: A Survey and Future Directions

• URL: http://arxiv.org/abs/2106.03727v1
• Date: Mon, 7 Jun 2021 15:42:36 GMT
• Title: Deep Neural Network-based Enhancement for Image and Video Streaming Systems: A Survey and Future Directions
• Authors: Royson Lee, Stylianos I. Venieris, Nicholas D. Lane
• Abstract summary: Deep learning has led to unprecedented performance in generating high-quality images from low-quality ones. We present state-of-the-art content delivery systems that employ neural enhancement as a key component in achieving both fast response time and high visual quality.
• Score: 20.835654670825782
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Internet-enabled smartphones and ultra-wide displays are transforming a variety of visual apps spanning from on-demand movies and 360{\deg} videos to video-conferencing and live streaming. However, robustly delivering visual content under fluctuating networking conditions on devices of diverse capabilities remains an open problem. In recent years, advances in the field of deep learning on tasks such as super-resolution and image enhancement have led to unprecedented performance in generating high-quality images from low-quality ones, a process we refer to as neural enhancement. In this paper, we survey state-of-the-art content delivery systems that employ neural enhancement as a key component in achieving both fast response time and high visual quality. We first present the components and architecture of existing content delivery systems, highlighting their challenges and motivating the use of neural enhancement models as a countermeasure. We then cover the deployment challenges of these models and analyze existing systems and their design decisions in efficiently overcoming these technical challenges. Additionally, we underline the key trends and common approaches across systems that target diverse use-cases. Finally, we present promising future directions based on the latest insights from deep learning research to further boost the quality of experience of content delivery systems.

Related papers

• Spatially Visual Perception for End-to-End Robotic Learning [33.490603706207075]
  We introduce a video-based spatial perception framework that leverages 3D spatial representations to address environmental variability. Our approach integrates a novel image augmentation technique, AugBlender, with a state-of-the-art monocular depth estimation model trained on internet-scale data.
  arXiv  Detail & Related papers  (2024-11-26T14:23:42Z)
• Transformer-based Image and Video Inpainting: Current Challenges and Future Directions [5.2088618044533215]
  Inpainting is a viable solution for various applications, including photographic restoration, video editing, and medical imaging. CNNs and generative adversarial networks (GANs) have significantly enhanced the inpainting task. Visual transformers have been exploited and offer some improvements to image or video inpainting.
  arXiv  Detail & Related papers  (2024-06-28T20:42:36Z)
• Chain-of-Spot: Interactive Reasoning Improves Large Vision-Language Models [81.71651422951074]
  Chain-of-Spot (CoS) method is a novel approach that enhances feature extraction by focusing on key regions of interest. This technique allows LVLMs to access more detailed visual information without altering the original image resolution. Our empirical findings demonstrate a significant improvement in LVLMs' ability to understand and reason about visual content.
  arXiv  Detail & Related papers  (2024-03-19T17:59:52Z)
• E2HQV: High-Quality Video Generation from Event Camera via Theory-Inspired Model-Aided Deep Learning [53.63364311738552]
  Bio-inspired event cameras or dynamic vision sensors are capable of capturing per-pixel brightness changes (called event-streams) in high temporal resolution and high dynamic range. It calls for events-to-video (E2V) solutions which take event-streams as input and generate high quality video frames for intuitive visualization. We propose textbfE2HQV, a novel E2V paradigm designed to produce high-quality video frames from events.
  arXiv  Detail & Related papers  (2024-01-16T05:10:50Z)
• A Survey on Super Resolution for video Enhancement Using GAN [0.0]
  Recent developments in super-resolution image and video using deep learning algorithms such as Generative Adversarial Networks are covered. Advancements aim to increase the visual clarity and quality of low-resolution video, have tremendous potential in a variety of sectors ranging from surveillance technology to medical imaging. This collection delves into the wider field of Generative Adversarial Networks, exploring their principles, training approaches, and applications across a broad range of domains.
  arXiv  Detail & Related papers  (2023-12-27T08:41:38Z)
• Integration and Performance Analysis of Artificial Intelligence and Computer Vision Based on Deep Learning Algorithms [5.734290974917728]
  This paper focuses on the analysis of the application effectiveness of the integration of deep learning and computer vision technologies. Deep learning achieves a historic breakthrough by constructing hierarchical neural networks, enabling end-to-end feature learning and semantic understanding of images. The successful experiences in the field of computer vision provide strong support for training deep learning algorithms.
  arXiv  Detail & Related papers  (2023-12-20T09:37:06Z)
• Neural Architecture Search for Dense Prediction Tasks in Computer Vision [74.9839082859151]
  Deep learning has led to a rising demand for neural network architecture engineering. neural architecture search (NAS) aims at automatically designing neural network architectures in a data-driven manner rather than manually. NAS has become applicable to a much wider range of problems in computer vision.
  arXiv  Detail & Related papers  (2022-02-15T08:06:50Z)
• Video Summarization Using Deep Neural Networks: A Survey [72.98424352264904]
  Video summarization technologies aim to create a concise and complete synopsis by selecting the most informative parts of the video content. This work focuses on the recent advances in the area and provides a comprehensive survey of the existing deep-learning-based methods for generic video summarization.
  arXiv  Detail & Related papers  (2021-01-15T11:41:29Z)
• Transformers in Vision: A Survey [101.07348618962111]
  Transformers enable modeling long dependencies between input sequence elements and support parallel processing of sequence. Transformers require minimal inductive biases for their design and are naturally suited as set-functions. This survey aims to provide a comprehensive overview of the Transformer models in the computer vision discipline.
  arXiv  Detail & Related papers  (2021-01-04T18:57:24Z)
• Neural Enhancement in Content Delivery Systems: The State-of-the-Art and Future Directions [16.04084457087104]
  Deep learning has led to unprecedented performance in generating high-quality images from low-quality ones. We present state-of-the-art content delivery systems that employ neural enhancement as a key component in achieving both fast response time and high visual quality.
  arXiv  Detail & Related papers  (2020-10-12T16:41:29Z)
• Learning Enriched Features for Real Image Restoration and Enhancement [166.17296369600774]
  convolutional neural networks (CNNs) have achieved dramatic improvements over conventional approaches for image restoration task. We present a novel architecture with the collective goals of maintaining spatially-precise high-resolution representations through the entire network. Our approach learns an enriched set of features that combines contextual information from multiple scales, while simultaneously preserving the high-resolution spatial details.
  arXiv  Detail & Related papers  (2020-03-15T11:04:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.