Beyond NeRF Underwater: Learning Neural Reflectance Fields for True Color Correction of Marine Imagery

• URL: http://arxiv.org/abs/2304.03384v2
• Date: Wed, 30 Aug 2023 22:20:50 GMT
• Title: Beyond NeRF Underwater: Learning Neural Reflectance Fields for True Color Correction of Marine Imagery
• Authors: Tianyi Zhang and Matthew Johnson-Roberson
• Abstract summary: Underwater imagery often exhibits distorted coloration as a result of light-water interactions. We propose an algorithm to restore the true color (albedo) in underwater imagery by jointly learning the effects of the medium and neural scene representations.
• Score: 16.16700041031569
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Underwater imagery often exhibits distorted coloration as a result of light-water interactions, which complicates the study of benthic environments in marine biology and geography. In this research, we propose an algorithm to restore the true color (albedo) in underwater imagery by jointly learning the effects of the medium and neural scene representations. Our approach models water effects as a combination of light attenuation with distance and backscattered light. The proposed neural scene representation is based on a neural reflectance field model, which learns albedos, normals, and volume densities of the underwater environment. We introduce a logistic regression model to separate water from the scene and apply distinct light physics during training. Our method avoids the need to estimate complex backscatter effects in water by employing several approximations, enhancing sampling efficiency and numerical stability during training. The proposed technique integrates underwater light effects into a volume rendering framework with end-to-end differentiability. Experimental results on both synthetic and real-world data demonstrate that our method effectively restores true color from underwater imagery, outperforming existing approaches in terms of color consistency.

Related papers

• Physics Informed and Data Driven Simulation of Underwater Images via Residual Learning [5.095097384893417]
  In general, underwater images suffer from color distortion and low contrast, because light is attenuated and backscattered as it propagates through water. An existing simple degradation model (similar to atmospheric image "hazing" effects) is not sufficient to properly represent the underwater image degradation. We propose a deep learning-based architecture to automatically simulate the underwater effects.
  arXiv  Detail & Related papers  (2024-02-07T21:53:28Z)
• A deep learning approach for marine snow synthesis and removal [55.86191108738564]
  This paper proposes a novel method to reduce the marine snow interference using deep learning techniques. We first synthesize realistic marine snow samples by training a Generative Adversarial Network (GAN) model. We then train a U-Net model to perform marine snow removal as an image to image translation task.
  arXiv  Detail & Related papers  (2023-11-27T07:19:41Z)
• Dual Adversarial Resilience for Collaborating Robust Underwater Image Enhancement and Perception [54.672052775549]
  In this work, we introduce a collaborative adversarial resilience network, dubbed CARNet, for underwater image enhancement and subsequent detection tasks. We propose a synchronized attack training strategy with both visual-driven and perception-driven attacks enabling the network to discern and remove various types of attacks. Experiments demonstrate that the proposed method outputs visually appealing enhancement images and perform averagely 6.71% higher detection mAP than state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-09-03T06:52:05Z)
• WaterFlow: Heuristic Normalizing Flow for Underwater Image Enhancement and Beyond [52.27796682972484]
  Existing underwater image enhancement methods mainly focus on image quality improvement, ignoring the effect on practice. We propose a normalizing flow for detection-driven underwater image enhancement, dubbed WaterFlow. Considering the differentiability and interpretability, we incorporate the prior into the data-driven mapping procedure.
  arXiv  Detail & Related papers  (2023-08-02T04:17:35Z)
• PUGAN: Physical Model-Guided Underwater Image Enhancement Using GAN with Dual-Discriminators [120.06891448820447]
  How to obtain clear and visually pleasant images has become a common concern of people. The task of underwater image enhancement (UIE) has also emerged as the times require. In this paper, we propose a physical model-guided GAN model for UIE, referred to as PUGAN. Our PUGAN outperforms state-of-the-art methods in both qualitative and quantitative metrics.
  arXiv  Detail & Related papers  (2023-06-15T07:41:12Z)
• TensoIR: Tensorial Inverse Rendering [51.57268311847087]
  TensoIR is a novel inverse rendering approach based on tensor factorization and neural fields. TensoRF is a state-of-the-art approach for radiance field modeling.
  arXiv  Detail & Related papers  (2023-04-24T21:39:13Z)
• MetaUE: Model-based Meta-learning for Underwater Image Enhancement [25.174894007563374]
  This paper proposes a model-based deep learning method for restoring clean images under various underwater scenarios. The meta-learning strategy is used to obtain a pre-trained model on the synthetic underwater dataset. The model is then fine-tuned on real underwater datasets to obtain a reliable underwater image enhancement model, called MetaUE.
  arXiv  Detail & Related papers  (2023-03-12T02:38:50Z)
• WaterNeRF: Neural Radiance Fields for Underwater Scenes [6.161668246821327]
  We advance state-of-the-art in neural radiance fields (NeRFs) to enable physics-informed dense depth estimation and color correction. Our proposed method, WaterNeRF, estimates parameters of a physics-based model for underwater image formation. We can produce novel views of degraded as well as corrected underwater images, along with dense depth of the scene.
  arXiv  Detail & Related papers  (2022-09-27T00:53:26Z)
• Underwater enhancement based on a self-learning strategy and attention mechanism for high-intensity regions [0.0]
  Images acquired during underwater activities suffer from environmental properties of the water, such as turbidity and light attenuation. Recent works related to underwater image enhancement, and based on deep learning approaches, tackle the lack of paired datasets generating synthetic ground-truth. We present a self-supervised learning methodology for underwater image enhancement based on deep learning that requires no paired datasets.
  arXiv  Detail & Related papers  (2022-08-04T19:55:40Z)
• Underwater Image Restoration via Contrastive Learning and a Real-world Dataset [59.35766392100753]
  We present a novel method for underwater image restoration based on unsupervised image-to-image translation framework. Our proposed method leveraged contrastive learning and generative adversarial networks to maximize the mutual information between raw and restored images.
  arXiv  Detail & Related papers  (2021-06-20T16:06:26Z)
• Domain Adaptive Adversarial Learning Based on Physics Model Feedback for Underwater Image Enhancement [10.143025577499039]
  We propose a new robust adversarial learning framework via physics model based feedback control and domain adaptation mechanism for enhancing underwater images. A new method for simulating underwater-like training dataset from RGB-D data by underwater image formation model is proposed. Final enhanced results on synthetic and real underwater images demonstrate the superiority of the proposed method.
  arXiv  Detail & Related papers  (2020-02-20T07:50:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.