Thermal Image Processing via Physics-Inspired Deep Networks

• URL: http://arxiv.org/abs/2108.07973v1
• Date: Wed, 18 Aug 2021 04:57:48 GMT
• Title: Thermal Image Processing via Physics-Inspired Deep Networks
• Authors: Vishwanath Saragadam, Akshat Dave, Ashok Veeraraghavan, Richard Baraniuk
• Abstract summary: DeepIR combines physically accurate sensor modeling with deep network-based image representation. DeepIR requires neither training data nor periodic ground-truth calibration with a known black body target. Simulated and real data experiments demonstrate that DeepIR can perform high-quality non-uniformity correction with as few as three images.
• Score: 21.094006629684376
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce DeepIR, a new thermal image processing framework that combines physically accurate sensor modeling with deep network-based image representation. Our key enabling observations are that the images captured by thermal sensors can be factored into slowly changing, scene-independent sensor non-uniformities (that can be accurately modeled using physics) and a scene-specific radiance flux (that is well-represented using a deep network-based regularizer). DeepIR requires neither training data nor periodic ground-truth calibration with a known black body target--making it well suited for practical computer vision tasks. We demonstrate the power of going DeepIR by developing new denoising and super-resolution algorithms that exploit multiple images of the scene captured with camera jitter. Simulated and real data experiments demonstrate that DeepIR can perform high-quality non-uniformity correction with as few as three images, achieving a 10dB PSNR improvement over competing approaches.

Related papers

• Robust Depth Enhancement via Polarization Prompt Fusion Tuning [112.88371907047396]
  We present a framework that leverages polarization imaging to improve inaccurate depth measurements from various depth sensors. Our method first adopts a learning-based strategy where a neural network is trained to estimate a dense and complete depth map from polarization data and a sensor depth map from different sensors. To further improve the performance, we propose a Polarization Prompt Fusion Tuning (PPFT) strategy to effectively utilize RGB-based models pre-trained on large-scale datasets.
  arXiv  Detail & Related papers  (2024-04-05T17:55:33Z)
• Learning to See Through Dazzle [0.0]
  Machine vision is susceptible to laser dazzle, where intense laser light can blind and distort its perception of the environment through oversaturation or permanent damage to sensor pixels. Here we employ a wavefront-coded phase mask to diffuse the energy of laser light and introduce a sandwich generative adversarial network (SGAN) to restore images from complex image degradations.
  arXiv  Detail & Related papers  (2024-02-24T22:22:02Z)
• Self-Supervised High Dynamic Range Imaging with Multi-Exposure Images in Dynamic Scenes [58.66427721308464]
  Self is a self-supervised reconstruction method that only requires dynamic multi-exposure images during training. Self achieves superior results against the state-of-the-art self-supervised methods, and comparable performance to supervised ones.
  arXiv  Detail & Related papers  (2023-10-03T07:10:49Z)
• Weakly-Supervised Optical Flow Estimation for Time-of-Flight [11.496094830445054]
  We propose a training algorithm, which allows to supervise Optical Flow networks directly on the reconstructed depth. We demonstrate that this approach enables the training of OF networks to align raw iToF measurements and compensate motion artifacts in the iToF depth images.
  arXiv  Detail & Related papers  (2022-10-11T09:47:23Z)
• DeepRM: Deep Recurrent Matching for 6D Pose Refinement [77.34726150561087]
  DeepRM is a novel recurrent network architecture for 6D pose refinement. The architecture incorporates LSTM units to propagate information through each refinement step. DeepRM achieves state-of-the-art performance on two widely accepted challenging datasets.
  arXiv  Detail & Related papers  (2022-05-28T16:18:08Z)
• GenISP: Neural ISP for Low-Light Machine Cognition [19.444297600977546]
  In low-light conditions, object detectors using raw image data are more robust than detectors using image data processed by an ISP pipeline. We propose a minimal neural ISP pipeline for machine cognition, named GenISP, that explicitly incorporates Color Space Transformation to a device-independent color space.
  arXiv  Detail & Related papers  (2022-05-07T17:17:24Z)
• Lightweight HDR Camera ISP for Robust Perception in Dynamic Illumination Conditions via Fourier Adversarial Networks [35.532434169432776]
  We propose a lightweight two-stage image enhancement algorithm sequentially balancing illumination and noise removal. We also propose a Fourier spectrum-based adversarial framework (AFNet) for consistent image enhancement under varying illumination conditions. Based on quantitative and qualitative evaluations, we also examine the practicality and effects of image enhancement techniques on the performance of common perception tasks.
  arXiv  Detail & Related papers  (2022-04-04T18:48:51Z)
• MantissaCam: Learning Snapshot High-dynamic-range Imaging with Perceptually-based In-pixel Irradiance Encoding [39.78877654934457]
  High-dynamic-range ( HDR) images are crucial in many computer vision applications. Here, we design a neural network-based algorithm that outperforms previous irradiance unwrapping methods. We show preliminary results of a prototype MantissaCam implemented with a programmable sensor.
  arXiv  Detail & Related papers  (2021-12-09T21:32:10Z)
• Learning Monocular Dense Depth from Events [53.078665310545745]
  Event cameras produce brightness changes in the form of a stream of asynchronous events instead of intensity frames. Recent learning-based approaches have been applied to event-based data, such as monocular depth prediction. We propose a recurrent architecture to solve this task and show significant improvement over standard feed-forward methods.
  arXiv  Detail & Related papers  (2020-10-16T12:36:23Z)
• Single-Image HDR Reconstruction by Learning to Reverse the Camera Pipeline [100.5353614588565]
  We propose to incorporate the domain knowledge of the LDR image formation pipeline into our model. We model the HDRto-LDR image formation pipeline as the (1) dynamic range clipping, (2) non-linear mapping from a camera response function, and (3) quantization. We demonstrate that the proposed method performs favorably against state-of-the-art single-image HDR reconstruction algorithms.
  arXiv  Detail & Related papers  (2020-04-02T17:59:04Z)
• Self-Supervised Linear Motion Deblurring [112.75317069916579]
  Deep convolutional neural networks are state-of-the-art for image deblurring. We present a differentiable reblur model for self-supervised motion deblurring. Our experiments demonstrate that self-supervised single image deblurring is really feasible.
  arXiv  Detail & Related papers  (2020-02-10T20:15:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.