TGTM: TinyML-based Global Tone Mapping for HDR Sensors
- URL: http://arxiv.org/abs/2405.05016v2
- Date: Fri, 10 May 2024 09:22:52 GMT
- Title: TGTM: TinyML-based Global Tone Mapping for HDR Sensors
- Authors: Peter Todorov, Julian Hartig, Jan Meyer-Siemon, Martin Fiedler, Gregor Schewior,
- Abstract summary: In this paper, we focus on HDR image tone mapping using a lightweight neural network applied on image histogram data.
Our proposed TinyML-based global tone mapping method, termed as TGTM, operates at 9,000 FLOPS per RGB image of any resolution.
Experimental results demonstrate that TGTM outperforms state-of-the-art methods on real HDR camera images by up to 5.85 dB higher PSNR with orders of magnitude less computations.
- Score: 0.0
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Advanced driver assistance systems (ADAS) relying on multiple cameras are increasingly prevalent in vehicle technology. Yet, conventional imaging sensors struggle to capture clear images in conditions with intense illumination contrast, such as tunnel exits, due to their limited dynamic range. Introducing high dynamic range (HDR) sensors addresses this issue. However, the process of converting HDR content to a displayable range via tone mapping often leads to inefficient computations, when performed directly on pixel data. In this paper, we focus on HDR image tone mapping using a lightweight neural network applied on image histogram data. Our proposed TinyML-based global tone mapping method, termed as TGTM, operates at 9,000 FLOPS per RGB image of any resolution. Additionally, TGTM offers a generic approach that can be incorporated to any classical tone mapping method. Experimental results demonstrate that TGTM outperforms state-of-the-art methods on real HDR camera images by up to 5.85 dB higher PSNR with orders of magnitude less computations.
Related papers
- HDRT: Infrared Capture for HDR Imaging [8.208995723545502]
We propose a new approach, High Dynamic Range Thermal (HDRT), for HDR acquisition using a separate, commonly available, thermal infrared (IR) sensor.
We propose a novel deep neural method (HDRTNet) which combines IR and SDR content to generate HDR images.
We show substantial quantitative and qualitative quality improvements on both over- and under-exposed images, showing that our approach is robust to capturing in multiple different lighting conditions.
arXiv Detail & Related papers (2024-06-08T13:43:44Z) - HDR-GS: Efficient High Dynamic Range Novel View Synthesis at 1000x Speed via Gaussian Splatting [76.5908492298286]
Existing HDR NVS methods are mainly based on NeRF.
Existing HDR NVS methods are mainly based on NeRF. They suffer from long training time and slow inference speed.
We propose a new framework, High Dynamic Range Gaussian Splatting (-GS), which can efficiently render novel HDR views.
arXiv Detail & Related papers (2024-05-24T00:46:58Z) - Simple Image Signal Processing using Global Context Guidance [56.41827271721955]
Deep learning-based ISPs aim to transform RAW images into DSLR-like RGB images using deep neural networks.
We propose a novel module that can be integrated into any neural ISP to capture the global context information from the full RAW images.
Our model achieves state-of-the-art results on different benchmarks using diverse and real smartphone images.
arXiv Detail & Related papers (2024-04-17T17:11:47Z) - Towards High-quality HDR Deghosting with Conditional Diffusion Models [88.83729417524823]
High Dynamic Range (LDR) images can be recovered from several Low Dynamic Range (LDR) images by existing Deep Neural Networks (DNNs) techniques.
DNNs still generate ghosting artifacts when LDR images have saturation and large motion.
We formulate the HDR deghosting problem as an image generation that leverages LDR features as the diffusion model's condition.
arXiv Detail & Related papers (2023-11-02T01:53:55Z) - Self-Supervised High Dynamic Range Imaging with Multi-Exposure Images in
Dynamic Scenes [58.66427721308464]
Self is a self-supervised reconstruction method that only requires dynamic multi-exposure images during training.
Self achieves superior results against the state-of-the-art self-supervised methods, and comparable performance to supervised ones.
arXiv Detail & Related papers (2023-10-03T07:10:49Z) - Lightweight Improved Residual Network for Efficient Inverse Tone Mapping [30.049931061503276]
Most media images on the internet remain in 8-bit standard dynamic range (SDR) format.
Inverse tone mapping (ITM) is crucial to unlock the full potential of abundant media images.
We propose a lightweight Improved Residual Network (IRNet) by enhancing the power of popular residual block for efficient ITM.
arXiv Detail & Related papers (2023-07-08T15:43:49Z) - Joint tone mapping and denoising of thermal infrared images via
multi-scale Retinex and multi-task learning [6.469120003158514]
Tone mapping algorithms for thermal infrared images with 16 bpp are investigated.
An optimized multi-scale Retinex algorithm is approximated with a deep learning approach based on the popular U-Net architecture.
The remaining noise in the images after tone mapping is reduced implicitly by utilizing a self-supervised deep learning approach.
arXiv Detail & Related papers (2023-05-01T07:14:32Z) - GlowGAN: Unsupervised Learning of HDR Images from LDR Images in the Wild [74.52723408793648]
We present the first method for learning a generative model of HDR images from in-the-wild LDR image collections in a fully unsupervised manner.
The key idea is to train a generative adversarial network (GAN) to generate HDR images which, when projected to LDR under various exposures, are indistinguishable from real LDR images.
Experiments show that our method GlowGAN can synthesize photorealistic HDR images in many challenging cases such as landscapes, lightning, or windows.
arXiv Detail & Related papers (2022-11-22T15:42:08Z) - MantissaCam: Learning Snapshot High-dynamic-range Imaging with
Perceptually-based In-pixel Irradiance Encoding [39.78877654934457]
High-dynamic-range ( HDR) images are crucial in many computer vision applications.
Here, we design a neural network-based algorithm that outperforms previous irradiance unwrapping methods.
We show preliminary results of a prototype MantissaCam implemented with a programmable sensor.
arXiv Detail & Related papers (2021-12-09T21:32:10Z) - Wavelet-Based Network For High Dynamic Range Imaging [64.66969585951207]
Existing methods, such as optical flow based and end-to-end deep learning based solutions, are error-prone either in detail restoration or ghosting artifacts removal.
In this work, we propose a novel frequency-guided end-to-end deep neural network (FNet) to conduct HDR fusion in the frequency domain, and Wavelet Transform (DWT) is used to decompose inputs into different frequency bands.
The low-frequency signals are used to avoid specific ghosting artifacts, while the high-frequency signals are used for preserving details.
arXiv Detail & Related papers (2021-08-03T12:26:33Z) - The Modulo Radon Transform: Theory, Algorithms and Applications [12.055044544892903]
We suggest a new model based on the Modulo Radon Transform (MRT), which we rigorously introduce and analyze.
By harnessing a joint design between hardware and algorithms, we present a single-shot HDR tomography approach.
Our solution paves a path for HDR acquisition in a number of related imaging problems.
arXiv Detail & Related papers (2021-05-10T08:38:48Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.