Atlantis: Enabling Underwater Depth Estimation with Stable Diffusion

• URL: http://arxiv.org/abs/2312.12471v1
• Date: Tue, 19 Dec 2023 08:56:33 GMT
• Title: Atlantis: Enabling Underwater Depth Estimation with Stable Diffusion
• Authors: Fan Zhang, Shaodi You, Yu Li, Ying Fu
• Abstract summary: We propose a novel pipeline for generating underwater images using accurate terrestrial depth data. This approach facilitates the training of supervised models for underwater depth estimation. We introduce a unique Depth2Underwater ControlNet, trained on specially prepared Underwater, Depth, Text data triplets.
• Score: 30.122666238416716
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Monocular depth estimation has experienced significant progress on terrestrial images in recent years, largely due to deep learning advancements. However, it remains inadequate for underwater scenes, primarily because of data scarcity. Given the inherent challenges of light attenuation and backscattering in water, acquiring clear underwater images or precise depth information is notably difficult and costly. Consequently, learning-based approaches often rely on synthetic data or turn to unsupervised or self-supervised methods to mitigate this lack of data. Nonetheless, the performance of these methods is often constrained by the domain gap and looser constraints. In this paper, we propose a novel pipeline for generating photorealistic underwater images using accurate terrestrial depth data. This approach facilitates the training of supervised models for underwater depth estimation, effectively reducing the performance disparity between terrestrial and underwater environments. Contrary to prior synthetic datasets that merely apply style transfer to terrestrial images without altering the scene content, our approach uniquely creates vibrant, non-existent underwater scenes by leveraging terrestrial depth data through the innovative Stable Diffusion model. Specifically, we introduce a unique Depth2Underwater ControlNet, trained on specially prepared \{Underwater, Depth, Text\} data triplets, for this generation task. Our newly developed dataset enables terrestrial depth estimation models to achieve considerable improvements, both quantitatively and qualitatively, on unseen underwater images, surpassing their terrestrial pre-trained counterparts. Moreover, the enhanced depth accuracy for underwater scenes also aids underwater image restoration techniques that rely on depth maps, further demonstrating our dataset's utility. The dataset will be available at https://github.com/zkawfanx/Atlantis.

Related papers

• Enhancing Underwater Imaging with 4-D Light Fields: Dataset and Method [77.80712860663886]
  4-D light fields (LFs) enhance underwater imaging plagued by light absorption, scattering, and other challenges. We propose a progressive framework for underwater 4-D LF image enhancement and depth estimation. We construct the first 4-D LF-based underwater image dataset for quantitative evaluation and supervised training of learning-based methods.
  arXiv  Detail & Related papers  (2024-08-30T15:06:45Z)
• WaterMono: Teacher-Guided Anomaly Masking and Enhancement Boosting for Robust Underwater Self-Supervised Monocular Depth Estimation [4.909989222186828]
  We propose WaterMono, a novel framework for depth estimation and image enhancement. It incorporates the following key measures: (1) We present a Teacher-Guided Anomaly Mask to identify dynamic regions within the images; (2) We employ depth information combined with the Underwater Image Formation Model to generate enhanced images, which in turn contribute to the depth estimation task; and (3) We utilize a rotated distillation strategy to enhance the model's rotational robustness.
  arXiv  Detail & Related papers  (2024-06-19T08:49:45Z)
• Physics-Inspired Synthesized Underwater Image Dataset [9.959844922120528]
  PHISWID is a dataset tailored for enhancing underwater image processing through physics-inspired image synthesis. Our results reveal that even a basic U-Net architecture, when trained with PHISWID, substantially outperforms existing methods in underwater image enhancement. We intend to release PHISWID publicly, contributing a significant resource to the advancement of underwater imaging technology.
  arXiv  Detail & Related papers  (2024-04-05T10:23:10Z)
• Improving Underwater Visual Tracking With a Large Scale Dataset and Image Enhancement [70.2429155741593]
  This paper presents a new dataset and general tracker enhancement method for Underwater Visual Object Tracking (UVOT) It poses distinct challenges; the underwater environment exhibits non-uniform lighting conditions, low visibility, lack of sharpness, low contrast, camouflage, and reflections from suspended particles. We propose a novel underwater image enhancement algorithm designed specifically to boost tracking quality. The method has resulted in a significant performance improvement, of up to 5.0% AUC, of state-of-the-art (SOTA) visual trackers.
  arXiv  Detail & Related papers  (2023-08-30T07:41:26Z)
• An evaluation of deep learning models for predicting water depth evolution in urban floods [59.31940764426359]
  We compare different deep learning models for prediction of water depth at high spatial resolution. Deep learning models are trained to reproduce the data simulated by the CADDIES cellular-automata flood model. Our results show that the deep learning models present in general lower errors compared to the other methods.
  arXiv  Detail & Related papers  (2023-02-20T16:08:54Z)
• Unpaired Overwater Image Defogging Using Prior Map Guided CycleGAN [60.257791714663725]
  We propose a Prior map Guided CycleGAN (PG-CycleGAN) for defogging of images with overwater scenes. The proposed method outperforms the state-of-the-art supervised, semi-supervised, and unsupervised defogging approaches.
  arXiv  Detail & Related papers  (2022-12-23T03:00:28Z)
• Self-Supervised Monocular Depth Underwater [8.830479021890575]
  In the past years estimation of depth from monocular images have shown great improvement. In the underwater environment results are still lagging behind due to appearance changes caused by the medium. We suggest several additions to the self-supervised framework to cope with the underwater environment.
  arXiv  Detail & Related papers  (2022-10-06T20:57:58Z)
• WaterNeRF: Neural Radiance Fields for Underwater Scenes [6.161668246821327]
  We advance state-of-the-art in neural radiance fields (NeRFs) to enable physics-informed dense depth estimation and color correction. Our proposed method, WaterNeRF, estimates parameters of a physics-based model for underwater image formation. We can produce novel views of degraded as well as corrected underwater images, along with dense depth of the scene.
  arXiv  Detail & Related papers  (2022-09-27T00:53:26Z)
• Underwater Image Restoration via Contrastive Learning and a Real-world Dataset [59.35766392100753]
  We present a novel method for underwater image restoration based on unsupervised image-to-image translation framework. Our proposed method leveraged contrastive learning and generative adversarial networks to maximize the mutual information between raw and restored images.
  arXiv  Detail & Related papers  (2021-06-20T16:06:26Z)
• Progressive Depth Learning for Single Image Dehazing [56.71963910162241]
  Existing dehazing methods often ignore the depth cues and fail in distant areas where heavier haze disturbs the visibility. We propose a deep end-to-end model that iteratively estimates image depths and transmission maps. Our approach benefits from explicitly modeling the inner relationship of image depth and transmission map, which is especially effective for distant hazy areas.
  arXiv  Detail & Related papers  (2021-02-21T05:24:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.