Evaluating Foveated Video Quality Using Entropic Differencing
- URL: http://arxiv.org/abs/2106.06817v1
- Date: Sat, 12 Jun 2021 16:29:13 GMT
- Title: Evaluating Foveated Video Quality Using Entropic Differencing
- Authors: Yize Jin, Anjul Patney, Alan Bovik
- Abstract summary: We propose a full reference (FR) foveated image quality assessment algorithm, which employs the natural scene statistics of bandpass responses.
We evaluate the proposed algorithm by measuring the correlations of the predictions that FED makes against human judgements.
The performance of the proposed algorithm yields state-of-the-art as compared with other existing full reference algorithms.
- Score: 1.5877673959068452
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Virtual Reality is regaining attention due to recent advancements in hardware
technology. Immersive images / videos are becoming widely adopted to carry
omnidirectional visual information. However, due to the requirements for higher
spatial and temporal resolution of real video data, immersive videos require
significantly larger bandwidth consumption. To reduce stresses on bandwidth,
foveated video compression is regaining popularity, whereby the space-variant
spatial resolution of the retina is exploited. Towards advancing the progress
of foveated video compression, we propose a full reference (FR) foveated image
quality assessment algorithm, which we call foveated entropic differencing
(FED), which employs the natural scene statistics of bandpass responses by
applying differences of local entropies weighted by a foveation-based error
sensitivity function. We evaluate the proposed algorithm by measuring the
correlations of the predictions that FED makes against human judgements on the
newly created 2D and 3D LIVE-FBT-FCVR databases for Virtual Reality (VR). The
performance of the proposed algorithm yields state-of-the-art as compared with
other existing full reference algorithms. Software for FED has been made
available at: http://live.ece.utexas.edu/research/Quality/FED.zip
Related papers
- Exploring Long- and Short-Range Temporal Information for Learned Video
Compression [54.91301930491466]
We focus on exploiting the unique characteristics of video content and exploring temporal information to enhance compression performance.
For long-range temporal information exploitation, we propose temporal prior that can update continuously within the group of pictures (GOP) during inference.
In that case temporal prior contains valuable temporal information of all decoded images within the current GOP.
In detail, we design a hierarchical structure to achieve multi-scale compensation.
arXiv Detail & Related papers (2022-08-07T15:57:18Z) - Learned Video Compression via Heterogeneous Deformable Compensation
Network [78.72508633457392]
We propose a learned video compression framework via heterogeneous deformable compensation strategy (HDCVC) to tackle the problems of unstable compression performance.
More specifically, the proposed algorithm extracts features from the two adjacent frames to estimate content-Neighborhood heterogeneous deformable (HetDeform) kernel offsets.
Experimental results indicate that HDCVC achieves superior performance than the recent state-of-the-art learned video compression approaches.
arXiv Detail & Related papers (2022-07-11T02:31:31Z) - Foveation-based Deep Video Compression without Motion Search [43.70396515286677]
Foveation protocols are desirable since only a small portion of a video viewed in VR may be visible as a user gazes in any given direction.
We implement foveation by introducing a Foveation Generator Unit (FGU) that generates foveation masks which direct the allocation of bits.
Our new compression model, which we call the Foveated MOtionless VIdeo Codec (Foveated MOVI-Codec), is able to efficiently compress videos without computing motion.
arXiv Detail & Related papers (2022-03-30T17:30:17Z) - FAVER: Blind Quality Prediction of Variable Frame Rate Videos [47.951054608064126]
Video quality assessment (VQA) remains an important and challenging problem that affects many applications at the widest scales.
We propose a first-of-a-kind blind VQA model for evaluating HFR videos, which we dub the Framerate-Aware Video Evaluator w/o Reference (FAVER)
Our experiments on several HFR video quality datasets show that FAVER outperforms other blind VQA algorithms at a reasonable computational cost.
arXiv Detail & Related papers (2022-01-05T07:54:12Z) - Deep Two-Stream Video Inference for Human Body Pose and Shape Estimation [18.14237514372724]
We propose a new framework to generate 3D human pose and mesh from RGB videos.
We train a two-stream temporal network based on transformer to predict SMPL parameters.
The proposed algorithm is extensively evaluated on the Human3.6 and 3DPW datasets.
arXiv Detail & Related papers (2021-10-22T10:01:13Z) - Perceptual Learned Video Compression with Recurrent Conditional GAN [158.0726042755]
We propose a Perceptual Learned Video Compression (PLVC) approach with recurrent conditional generative adversarial network.
PLVC learns to compress video towards good perceptual quality at low bit-rate.
The user study further validates the outstanding perceptual performance of PLVC in comparison with the latest learned video compression approaches.
arXiv Detail & Related papers (2021-09-07T13:36:57Z) - FOVQA: Blind Foveated Video Quality Assessment [1.4127304025810108]
We develop a no-reference (NR) foveated video quality assessment model, called FOVQA.
It is based on new models of space-variant natural scene statistics (NSS) and natural video statistics (NVS)
FOVQA achieves state-of-the-art (SOTA) performance on the new 2D LIVE-FBT-FCVR database.
arXiv Detail & Related papers (2021-06-24T21:38:22Z) - Decomposition, Compression, and Synthesis (DCS)-based Video Coding: A
Neural Exploration via Resolution-Adaptive Learning [30.54722074562783]
We decompose the input video into respective spatial texture frames (STF) at its native spatial resolution.
Then, we compress them together using any popular video coder.
Finally, we synthesize decoded STFs and TMFs for high-quality video reconstruction at the same resolution as its native input.
arXiv Detail & Related papers (2020-12-01T17:23:53Z) - Robust Unsupervised Video Anomaly Detection by Multi-Path Frame
Prediction [61.17654438176999]
We propose a novel and robust unsupervised video anomaly detection method by frame prediction with proper design.
Our proposed method obtains the frame-level AUROC score of 88.3% on the CUHK Avenue dataset.
arXiv Detail & Related papers (2020-11-05T11:34:12Z) - ST-GREED: Space-Time Generalized Entropic Differences for Frame Rate
Dependent Video Quality Prediction [63.749184706461826]
We study how perceptual quality is affected by frame rate, and how frame rate and compression combine to affect perceived quality.
We devise an objective VQA model called Space-Time GeneRalized Entropic Difference (GREED) which analyzes the statistics of spatial and temporal band-pass video coefficients.
GREED achieves state-of-the-art performance on the LIVE-YT-HFR Database when compared with existing VQA models.
arXiv Detail & Related papers (2020-10-26T16:54:33Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.