DEHRFormer: Real-time Transformer for Depth Estimation and Haze Removal from Varicolored Haze Scenes

• URL: http://arxiv.org/abs/2303.06905v1
• Date: Mon, 13 Mar 2023 07:47:18 GMT
• Title: DEHRFormer: Real-time Transformer for Depth Estimation and Haze Removal from Varicolored Haze Scenes
• Authors: Sixiang Chen, Tian Ye, Jun Shi, Yun Liu, JingXia Jiang, Erkang Chen, Peng Chen
• Abstract summary: We propose a real-time transformer for simultaneous single image Depth Estimation and Haze Removal. DEHRFormer consists of a single encoder and two task-specific decoders. We introduce a novel learning paradigm that utilizes contrastive learning and domain consistency learning to tackle weak-generalization problem for real-world dehazing.
• Score: 10.174140482558904
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Varicolored haze caused by chromatic casts poses haze removal and depth estimation challenges. Recent learning-based depth estimation methods are mainly targeted at dehazing first and estimating depth subsequently from haze-free scenes. This way, the inner connections between colored haze and scene depth are lost. In this paper, we propose a real-time transformer for simultaneous single image Depth Estimation and Haze Removal (DEHRFormer). DEHRFormer consists of a single encoder and two task-specific decoders. The transformer decoders with learnable queries are designed to decode coupling features from the task-agnostic encoder and project them into clean image and depth map, respectively. In addition, we introduce a novel learning paradigm that utilizes contrastive learning and domain consistency learning to tackle weak-generalization problem for real-world dehazing, while predicting the same depth map from the same scene with varicolored haze. Experiments demonstrate that DEHRFormer achieves significant performance improvement across diverse varicolored haze scenes over previous depth estimation networks and dehazing approaches.

Related papers

• Back to the Color: Learning Depth to Specific Color Transformation for Unsupervised Depth Estimation [45.07558105128673]
  discrepancies between synthetic and real-world colors pose significant challenges for depth estimation in real-world scenes. We propose Back2Color, a framework that predicts realistic colors from depth using a model trained on real-world data. We also present VADepth, based on the Vision Attention Network, which offers lower computational complexity and higher accuracy than transformers.
  arXiv  Detail & Related papers  (2024-06-11T21:55:20Z)
• Unveiling the Depths: A Multi-Modal Fusion Framework for Challenging Scenarios [103.72094710263656]
  This paper presents a novel approach that identifies and integrates dominant cross-modality depth features with a learning-based framework. We propose a novel confidence loss steering a confidence predictor network to yield a confidence map specifying latent potential depth areas. With the resulting confidence map, we propose a multi-modal fusion network that fuses the final depth in an end-to-end manner.
  arXiv  Detail & Related papers  (2024-02-19T04:39:16Z)
• Learning to Adapt CLIP for Few-Shot Monocular Depth Estimation [31.34615135846137]
  We propose a few-shot-based method which learns to adapt the Vision-Language Models for monocular depth estimation. Specifically, it assigns different depth bins for different scenes, which can be selected by the model during inference. With only one image per scene for training, our extensive experiment results on the NYU V2 and KITTI dataset demonstrate that our method outperforms the previous state-of-the-art method by up to 10.6% in terms of MARE.
  arXiv  Detail & Related papers  (2023-11-02T06:56:50Z)
• Depth Estimation from a Single Optical Encoded Image using a Learned Colored-Coded Aperture [18.830374973687416]
  State-of-the-art approaches improve the discrimination between different depths by introducing a binary-coded aperture (CA) in the lens aperture. Color-coded apertures (CCA) can also produce color misalignment in the captured image which can be utilized to estimate disparity. We propose a CCA with a greater number of color filters and richer spectral information to optically encode relevant depth information in a single snapshot.
  arXiv  Detail & Related papers  (2023-09-14T21:30:55Z)
• SelfPromer: Self-Prompt Dehazing Transformers with Depth-Consistency [51.92434113232977]
  This work presents an effective depth-consistency self-prompt Transformer for image dehazing. It is motivated by an observation that the estimated depths of an image with haze residuals and its clear counterpart vary. By incorporating the prompt, prompt embedding, and prompt attention into an encoder-decoder network based on VQGAN, we can achieve better perception quality.
  arXiv  Detail & Related papers  (2023-03-13T11:47:24Z)
• Towards Accurate Reconstruction of 3D Scene Shape from A Single Monocular Image [91.71077190961688]
  We propose a two-stage framework that first predicts depth up to an unknown scale and shift from a single monocular image. We then exploits 3D point cloud data to predict the depth shift and the camera's focal length that allow us to recover 3D scene shapes. We test our depth model on nine unseen datasets and achieve state-of-the-art performance on zero-shot evaluation.
  arXiv  Detail & Related papers  (2022-08-28T16:20:14Z)
• End-to-end Learning for Joint Depth and Image Reconstruction from Diffracted Rotation [10.896567381206715]
  We propose a novel end-to-end learning approach for depth from diffracted rotation. Our approach requires a significantly less complex model and less training data, yet it is superior to existing methods in the task of monocular depth estimation.
  arXiv  Detail & Related papers  (2022-04-14T16:14:37Z)
• Progressive Depth Learning for Single Image Dehazing [56.71963910162241]
  Existing dehazing methods often ignore the depth cues and fail in distant areas where heavier haze disturbs the visibility. We propose a deep end-to-end model that iteratively estimates image depths and transmission maps. Our approach benefits from explicitly modeling the inner relationship of image depth and transmission map, which is especially effective for distant hazy areas.
  arXiv  Detail & Related papers  (2021-02-21T05:24:18Z)
• Dual Pixel Exploration: Simultaneous Depth Estimation and Image Restoration [77.1056200937214]
  We study the formation of the DP pair which links the blur and the depth information. We propose an end-to-end DDDNet (DP-based Depth and De Network) to jointly estimate the depth and restore the image.
  arXiv  Detail & Related papers  (2020-12-01T06:53:57Z)
• DiverseDepth: Affine-invariant Depth Prediction Using Diverse Data [110.29043712400912]
  We present a method for depth estimation with monocular images, which can predict high-quality depth on diverse scenes up to an affine transformation. Experiments show that our method outperforms previous methods on 8 datasets by a large margin with the zero-shot test setting.
  arXiv  Detail & Related papers  (2020-02-03T05:38:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.