Related papers: NAViDAd: A No-Reference Audio-Visual Quality Metric Based on a Deep Autoencoder

NAViDAd: A No-Reference Audio-Visual Quality Metric Based on a Deep Autoencoder

URL: http://arxiv.org/abs/2001.11406v2
Date: Tue, 4 Feb 2020 19:08:49 GMT
Title: NAViDAd: A No-Reference Audio-Visual Quality Metric Based on a Deep Autoencoder
Authors: Helard Martinez, M. C. Farias, A. Hines
Abstract summary: We propose a No-Reference Audio-Visual Quality Metric Based on a Deep Autoencoder (NAViDAd) The model is formed by a 2-layer framework that includes a deep autoencoder layer and a classification layer. The model performed well when tested against the UnB-AV and the LiveNetflix-II databases.
Score: 0.0
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Abstract: The development of models for quality prediction of both audio and video signals is a fairly mature field. But, although several multimodal models have been proposed, the area of audio-visual quality prediction is still an emerging area. In fact, despite the reasonable performance obtained by combination and parametric metrics, currently there is no reliable pixel-based audio-visual quality metric. The approach presented in this work is based on the assumption that autoencoders, fed with descriptive audio and video features, might produce a set of features that is able to describe the complex audio and video interactions. Based on this hypothesis, we propose a No-Reference Audio-Visual Quality Metric Based on a Deep Autoencoder (NAViDAd). The model visual features are natural scene statistics (NSS) and spatial-temporal measures of the video component. Meanwhile, the audio features are obtained by computing the spectrogram representation of the audio component. The model is formed by a 2-layer framework that includes a deep autoencoder layer and a classification layer. These two layers are stacked and trained to build the deep neural network model. The model is trained and tested using a large set of stimuli, containing representative audio and video artifacts. The model performed well when tested against the UnB-AV and the LiveNetflix-II databases. %Results shows that this type of approach produces quality scores that are highly correlated to subjective quality scores.