Towards Efficient SDRTV-to-HDRTV by Learning from Image Formation

• URL: http://arxiv.org/abs/2309.04084v1
• Date: Fri, 8 Sep 2023 02:50:54 GMT
• Title: Towards Efficient SDRTV-to-HDRTV by Learning from Image Formation
• Authors: Xiangyu Chen, Zheyuan Li, Zhengwen Zhang, Jimmy S. Ren, Yihao Liu, Jingwen He, Yu Qiao, Jiantao Zhou, Chao Dong
• Abstract summary: Modern displays are capable of rendering video content with high dynamic range (WCG) and wide color gamut (SDR) The majority of available resources are still in standard dynamic range (SDR) We define and analyze the SDRTV-to-TV task by modeling the formation of SDRTV/TV content. Our method is primarily designed for ultra-high-definition TV content and is therefore effective and lightweight for processing 4K resolution images.
• Score: 51.26219245226384
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Modern displays are capable of rendering video content with high dynamic range (HDR) and wide color gamut (WCG). However, the majority of available resources are still in standard dynamic range (SDR). As a result, there is significant value in transforming existing SDR content into the HDRTV standard. In this paper, we define and analyze the SDRTV-to-HDRTV task by modeling the formation of SDRTV/HDRTV content. Our analysis and observations indicate that a naive end-to-end supervised training pipeline suffers from severe gamut transition errors. To address this issue, we propose a novel three-step solution pipeline called HDRTVNet++, which includes adaptive global color mapping, local enhancement, and highlight refinement. The adaptive global color mapping step uses global statistics as guidance to perform image-adaptive color mapping. A local enhancement network is then deployed to enhance local details. Finally, we combine the two sub-networks above as a generator and achieve highlight consistency through GAN-based joint training. Our method is primarily designed for ultra-high-definition TV content and is therefore effective and lightweight for processing 4K resolution images. We also construct a dataset using HDR videos in the HDR10 standard, named HDRTV1K that contains 1235 and 117 training images and 117 testing images, all in 4K resolution. Besides, we select five metrics to evaluate the results of SDRTV-to-HDRTV algorithms. Our final results demonstrate state-of-the-art performance both quantitatively and visually. The code, model and dataset are available at https://github.com/xiaom233/HDRTVNet-plus.

Related papers

• HDR-GS: Efficient High Dynamic Range Novel View Synthesis at 1000x Speed via Gaussian Splatting [76.5908492298286]
  Existing HDR NVS methods are mainly based on NeRF. They suffer from long training time and slow inference speed. We propose a new framework, High Dynamic Range Gaussian Splatting (-GS)
  arXiv  Detail & Related papers  (2024-05-24T00:46:58Z)
• Towards Real-World HDR Video Reconstruction: A Large-Scale Benchmark Dataset and A Two-Stage Alignment Network [16.39592423564326]
  Existing methods are mostly trained on synthetic datasets, which perform poorly in real scenes. We present Real-V, a large-scale real-world benchmark dataset for HDR video reconstruction.
  arXiv  Detail & Related papers  (2024-04-30T23:29:26Z)
• FastHDRNet: A new efficient method for SDR-to-HDR Translation [5.224011800476952]
  We propose a neural network for SDR to HDR conversion, termed "FastNet" The architecture is designed as a lightweight network that utilizes global statistics and local information with super high efficiency.
  arXiv  Detail & Related papers  (2024-04-06T03:25:24Z)
• Training Neural Networks on RAW and HDR Images for Restoration Tasks [59.41340420564656]
  In this work, we test approaches on three popular image restoration applications: denoising, deblurring, and single-image super-resolution. Our results indicate that neural networks train significantly better on HDR and RAW images represented in display color spaces. This small change to the training strategy can bring a very substantial gain in performance, up to 10-15 dB.
  arXiv  Detail & Related papers  (2023-12-06T17:47:16Z)
• Self-Supervised High Dynamic Range Imaging with Multi-Exposure Images in Dynamic Scenes [58.66427721308464]
  Self is a self-supervised reconstruction method that only requires dynamic multi-exposure images during training. Self achieves superior results against the state-of-the-art self-supervised methods, and comparable performance to supervised ones.
  arXiv  Detail & Related papers  (2023-10-03T07:10:49Z)
• HDR Video Reconstruction with a Large Dynamic Dataset in Raw and sRGB Domains [23.309488653045026]
  High dynamic range ( HDR) video reconstruction is attracting more and more attention due to the superior visual quality compared with those of low dynamic range (LDR) videos. There are still no real LDR- pairs for dynamic scenes due to the difficulty in capturing LDR- frames simultaneously. In this work, we propose to utilize a staggered sensor to capture two alternate exposure images simultaneously, which are then fused into an HDR frame in both raw and sRGB domains.
  arXiv  Detail & Related papers  (2023-04-10T11:59:03Z)
• Learning a Practical SDR-to-HDRTV Up-conversion using New Dataset and Degradation Models [4.0336006284433665]
  In media industry, the demand of SDR-to-TV up-conversion arises when users possess HDR-WCG (high dynamic range-wide color gamut) Current methods tend to produce dim and desaturated result, making nearly no improvement on viewing experience. We propose new HDRTV dataset (dubbed HDRTV4K) and new HDR-to-SDR models.
  arXiv  Detail & Related papers  (2023-03-23T04:40:33Z)
• A New Journey from SDRTV to HDRTV [36.58487005995048]
  We conduct an analysis of SDRTV-to-TV task by modeling the formation of SDRTV/TV content. We present a lightweight network that utilizes global statistics as guidance to conduct image-adaptive color mapping.
  arXiv  Detail & Related papers  (2021-08-18T05:17:08Z)
• AIM 2020 Challenge on Video Extreme Super-Resolution: Methods and Results [96.74919503142014]
  This paper reviews the video extreme super-resolution challenge associated with the AIM 2020 workshop at ECCV 2020. Track 1 is set up to gauge the state-of-the-art for such a demanding task, where fidelity to the ground truth is measured by PSNR and SSIM. Track 2 therefore aims at generating visually pleasing results, which are ranked according to human perception, evaluated by a user study.
  arXiv  Detail & Related papers  (2020-09-14T09:36:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.