Deep Neural Network-based Enhancement for Image and Video Streaming
Systems: A Survey and Future Directions
- URL: http://arxiv.org/abs/2106.03727v1
- Date: Mon, 7 Jun 2021 15:42:36 GMT
- Title: Deep Neural Network-based Enhancement for Image and Video Streaming
Systems: A Survey and Future Directions
- Authors: Royson Lee, Stylianos I. Venieris, Nicholas D. Lane
- Abstract summary: Deep learning has led to unprecedented performance in generating high-quality images from low-quality ones.
We present state-of-the-art content delivery systems that employ neural enhancement as a key component in achieving both fast response time and high visual quality.
- Score: 20.835654670825782
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Internet-enabled smartphones and ultra-wide displays are transforming a
variety of visual apps spanning from on-demand movies and 360{\deg} videos to
video-conferencing and live streaming. However, robustly delivering visual
content under fluctuating networking conditions on devices of diverse
capabilities remains an open problem. In recent years, advances in the field of
deep learning on tasks such as super-resolution and image enhancement have led
to unprecedented performance in generating high-quality images from low-quality
ones, a process we refer to as neural enhancement. In this paper, we survey
state-of-the-art content delivery systems that employ neural enhancement as a
key component in achieving both fast response time and high visual quality. We
first present the components and architecture of existing content delivery
systems, highlighting their challenges and motivating the use of neural
enhancement models as a countermeasure. We then cover the deployment challenges
of these models and analyze existing systems and their design decisions in
efficiently overcoming these technical challenges. Additionally, we underline
the key trends and common approaches across systems that target diverse
use-cases. Finally, we present promising future directions based on the latest
insights from deep learning research to further boost the quality of experience
of content delivery systems.
Related papers
- Transformer-based Image and Video Inpainting: Current Challenges and Future Directions [5.2088618044533215]
Inpainting is a viable solution for various applications, including photographic restoration, video editing, and medical imaging.
CNNs and generative adversarial networks (GANs) have significantly enhanced the inpainting task.
Visual transformers have been exploited and offer some improvements to image or video inpainting.
arXiv Detail & Related papers (2024-06-28T20:42:36Z) - Chain-of-Spot: Interactive Reasoning Improves Large Vision-Language Models [81.71651422951074]
Chain-of-Spot (CoS) method is a novel approach that enhances feature extraction by focusing on key regions of interest.
This technique allows LVLMs to access more detailed visual information without altering the original image resolution.
Our empirical findings demonstrate a significant improvement in LVLMs' ability to understand and reason about visual content.
arXiv Detail & Related papers (2024-03-19T17:59:52Z) - Reimagining Reality: A Comprehensive Survey of Video Inpainting
Techniques [6.36998581871295]
Video inpainting is a process that restores or fills in missing or corrupted portions of video sequences with plausible content.
Our study deconstructs major techniques, their underpinning theories, and their effective applications.
We employ a human-centric approach to assess visual quality, enlisting a panel of annotators to evaluate the output of different video inpainting techniques.
arXiv Detail & Related papers (2024-01-31T14:41:40Z) - E2HQV: High-Quality Video Generation from Event Camera via
Theory-Inspired Model-Aided Deep Learning [53.63364311738552]
Bio-inspired event cameras or dynamic vision sensors are capable of capturing per-pixel brightness changes (called event-streams) in high temporal resolution and high dynamic range.
It calls for events-to-video (E2V) solutions which take event-streams as input and generate high quality video frames for intuitive visualization.
We propose textbfE2HQV, a novel E2V paradigm designed to produce high-quality video frames from events.
arXiv Detail & Related papers (2024-01-16T05:10:50Z) - A Survey on Super Resolution for video Enhancement Using GAN [0.0]
Recent developments in super-resolution image and video using deep learning algorithms such as Generative Adversarial Networks are covered.
Advancements aim to increase the visual clarity and quality of low-resolution video, have tremendous potential in a variety of sectors ranging from surveillance technology to medical imaging.
This collection delves into the wider field of Generative Adversarial Networks, exploring their principles, training approaches, and applications across a broad range of domains.
arXiv Detail & Related papers (2023-12-27T08:41:38Z) - Integration and Performance Analysis of Artificial Intelligence and
Computer Vision Based on Deep Learning Algorithms [5.734290974917728]
This paper focuses on the analysis of the application effectiveness of the integration of deep learning and computer vision technologies.
Deep learning achieves a historic breakthrough by constructing hierarchical neural networks, enabling end-to-end feature learning and semantic understanding of images.
The successful experiences in the field of computer vision provide strong support for training deep learning algorithms.
arXiv Detail & Related papers (2023-12-20T09:37:06Z) - Neural Architecture Search for Dense Prediction Tasks in Computer Vision [74.9839082859151]
Deep learning has led to a rising demand for neural network architecture engineering.
neural architecture search (NAS) aims at automatically designing neural network architectures in a data-driven manner rather than manually.
NAS has become applicable to a much wider range of problems in computer vision.
arXiv Detail & Related papers (2022-02-15T08:06:50Z) - Video Summarization Using Deep Neural Networks: A Survey [72.98424352264904]
Video summarization technologies aim to create a concise and complete synopsis by selecting the most informative parts of the video content.
This work focuses on the recent advances in the area and provides a comprehensive survey of the existing deep-learning-based methods for generic video summarization.
arXiv Detail & Related papers (2021-01-15T11:41:29Z) - Transformers in Vision: A Survey [101.07348618962111]
Transformers enable modeling long dependencies between input sequence elements and support parallel processing of sequence.
Transformers require minimal inductive biases for their design and are naturally suited as set-functions.
This survey aims to provide a comprehensive overview of the Transformer models in the computer vision discipline.
arXiv Detail & Related papers (2021-01-04T18:57:24Z) - Neural Enhancement in Content Delivery Systems: The State-of-the-Art and
Future Directions [16.04084457087104]
Deep learning has led to unprecedented performance in generating high-quality images from low-quality ones.
We present state-of-the-art content delivery systems that employ neural enhancement as a key component in achieving both fast response time and high visual quality.
arXiv Detail & Related papers (2020-10-12T16:41:29Z) - Learning Enriched Features for Real Image Restoration and Enhancement [166.17296369600774]
convolutional neural networks (CNNs) have achieved dramatic improvements over conventional approaches for image restoration task.
We present a novel architecture with the collective goals of maintaining spatially-precise high-resolution representations through the entire network.
Our approach learns an enriched set of features that combines contextual information from multiple scales, while simultaneously preserving the high-resolution spatial details.
arXiv Detail & Related papers (2020-03-15T11:04:30Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.