On the Computation of BD-Rate over a Set of Videos for Fair Assessment of Performance of Learned Video Codecs
- URL: http://arxiv.org/abs/2409.08772v1
- Date: Fri, 13 Sep 2024 12:30:15 GMT
- Title: On the Computation of BD-Rate over a Set of Videos for Fair Assessment of Performance of Learned Video Codecs
- Authors: M. Akin Yilmaz, Onur Keleş, A. Murat Tekalp,
- Abstract summary: The Bjontegaard Delta (BD) measure is widely employed to evaluate and quantify the variations in the rate-distortion(RD) performance across different codecs.
We claim that the current practice in the learned video compression community of computing the average BD value over a dataset based on the average RD curve of multiple videos can lead to misleading conclusions.
- Score: 7.714092783675679
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The Bj{\o}ntegaard Delta (BD) measure is widely employed to evaluate and quantify the variations in the rate-distortion(RD) performance across different codecs. Many researchers report the average BD value over multiple videos within a dataset for different codecs. We claim that the current practice in the learned video compression community of computing the average BD value over a dataset based on the average RD curve of multiple videos can lead to misleading conclusions. We show both by analysis of a simplistic case of linear RD curves and experimental results with two recent learned video codecs that averaging RD curves can lead to a single video to disproportionately influence the average BD value especially when the operating bitrate range of different codecs do not exactly match. Instead, we advocate for calculating the BD measure per-video basis, as commonly done by the traditional video compression community, followed by averaging the individual BD values over videos, to provide a fair comparison of learned video codecs. Our experimental results demonstrate that the comparison of two recent learned video codecs is affected by how we evaluate the average BD measure.
Related papers
- Benchmarking Conventional and Learned Video Codecs with a Low-Delay Configuration [11.016119119250765]
This paper conducts a comparative study of state-of-the-art conventional and learned video coding methods based on a low delay configuration.
To allow a fair and meaningful comparison, the evaluation was performed on test sequences defined in the AOM and MPEG common test conditions in the YCbCr 4:2:0 color space.
The evaluation results show that the JVET ECM codecs offer the best overall coding performance among all codecs tested.
arXiv Detail & Related papers (2024-08-09T12:55:23Z) - Not All Pairs are Equal: Hierarchical Learning for Average-Precision-Oriented Video Retrieval [80.09819072780193]
Average Precision (AP) assesses the overall rankings of relevant videos at the top list.
Recent video retrieval methods utilize pair-wise losses that treat all sample pairs equally.
arXiv Detail & Related papers (2024-07-22T11:52:04Z) - Hierarchical B-frame Video Coding for Long Group of Pictures [42.229439873835254]
We present an end-to-end learned video for random access that combines training on long sequences of frames, rate allocation and content adaptation on inference.
Under common test conditions, it achieves results comparable to VTM in terms of YUV-PSNR BD-Rate on some classes of videos.
On average it surpasses open LD and RA end-to-end solutions in terms of VMAF and YUV BD-Rates.
arXiv Detail & Related papers (2024-06-24T11:29:52Z) - Towards Debiasing Frame Length Bias in Text-Video Retrieval via Causal
Intervention [72.12974259966592]
We present a unique and systematic study of a temporal bias due to frame length discrepancy between training and test sets of trimmed video clips.
We propose a causal debiasing approach and perform extensive experiments and ablation studies on the Epic-Kitchens-100, YouCook2, and MSR-VTT datasets.
arXiv Detail & Related papers (2023-09-17T15:58:27Z) - Video Compression with Arbitrary Rescaling Network [8.489428003916622]
We propose a rate-guided arbitrary rescaling network (RARN) for video resizing before encoding.
The lightweight RARN structure can process FHD (1080p) content at real-time speed (91 FPS) and obtain a considerable rate reduction.
arXiv Detail & Related papers (2023-06-07T07:15:18Z) - Video compression dataset and benchmark of learning-based video-quality
metrics [55.41644538483948]
We present a new benchmark for video-quality metrics that evaluates video compression.
It is based on a new dataset consisting of about 2,500 streams encoded using different standards.
Subjective scores were collected using crowdsourced pairwise comparisons.
arXiv Detail & Related papers (2022-11-22T09:22:28Z) - Perceptual Learned Video Compression with Recurrent Conditional GAN [158.0726042755]
We propose a Perceptual Learned Video Compression (PLVC) approach with recurrent conditional generative adversarial network.
PLVC learns to compress video towards good perceptual quality at low bit-rate.
The user study further validates the outstanding perceptual performance of PLVC in comparison with the latest learned video compression approaches.
arXiv Detail & Related papers (2021-09-07T13:36:57Z) - Objective video quality metrics application to video codecs comparisons:
choosing the best for subjective quality estimation [101.18253437732933]
Quality assessment plays a key role in creating and comparing video compression algorithms.
For comparison, we used a set of videos encoded with video codecs of different standards, and visual quality scores collected for the resulting set of streams since 2018 until 2021.
arXiv Detail & Related papers (2021-07-21T17:18:11Z) - Content Adaptive and Error Propagation Aware Deep Video Compression [110.31693187153084]
We propose a content adaptive and error propagation aware video compression system.
Our method employs a joint training strategy by considering the compression performance of multiple consecutive frames instead of a single frame.
Instead of using the hand-crafted coding modes in the traditional compression systems, we design an online encoder updating scheme in our system.
arXiv Detail & Related papers (2020-03-25T09:04:24Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.