Leveraging Scene Geometry and Depth Information for Robust Image Deraining

• URL: http://arxiv.org/abs/2412.19913v1
• Date: Fri, 27 Dec 2024 20:18:46 GMT
• Title: Leveraging Scene Geometry and Depth Information for Robust Image Deraining
• Authors: Ningning Xu, Jidong J. Yang,
• Abstract summary: Image deraining holds great potential for enhancing the vision of autonomous vehicles in rainy conditions. Previous works have primarily focused on employing a single network architecture to generate derained images. We introduce a novel learning framework that integrates multiple networks: an AutoEncoder for deraining, an auxiliary network to incorporate depth information, and two supervision networks to enforce feature consistency between rainy and clear scenes.
• Score: 0.9208007322096533
• License:
• Abstract: Image deraining holds great potential for enhancing the vision of autonomous vehicles in rainy conditions, contributing to safer driving. Previous works have primarily focused on employing a single network architecture to generate derained images. However, they often fail to fully exploit the rich prior knowledge embedded in the scenes. Particularly, most methods overlook the depth information that can provide valuable context about scene geometry and guide more robust deraining. In this work, we introduce a novel learning framework that integrates multiple networks: an AutoEncoder for deraining, an auxiliary network to incorporate depth information, and two supervision networks to enforce feature consistency between rainy and clear scenes. This multi-network design enables our model to effectively capture the underlying scene structure, producing clearer and more accurately derained images, leading to improved object detection for autonomous vehicles. Extensive experiments on three widely-used datasets demonstrated the effectiveness of our proposed method.

Related papers

• HVDistill: Transferring Knowledge from Images to Point Clouds via Unsupervised Hybrid-View Distillation [106.09886920774002]
  We present a hybrid-view-based knowledge distillation framework, termed HVDistill, to guide the feature learning of a point cloud neural network. Our method achieves consistent improvements over the baseline trained from scratch and significantly out- performs the existing schemes.
  arXiv  Detail & Related papers  (2024-03-18T14:18:08Z)
• NiteDR: Nighttime Image De-Raining with Cross-View Sensor Cooperative Learning for Dynamic Driving Scenes [49.92839157944134]
  In nighttime driving scenes, insufficient and uneven lighting shrouds the scenes in darkness, resulting degradation of image quality and visibility. We develop an image de-raining framework tailored for rainy nighttime driving scenes. It aims to remove rain artifacts, enrich scene representation, and restore useful information.
  arXiv  Detail & Related papers  (2024-02-28T09:02:33Z)
• Contrastive Learning Based Recursive Dynamic Multi-Scale Network for Image Deraining [47.764883957379745]
  Rain streaks significantly decrease the visibility of captured images. Existing deep learning-based image deraining methods employ manually crafted networks and learn a straightforward projection from rainy images to clear images. We propose a contrastive learning-based image deraining method that investigates the correlation between rainy and clear images.
  arXiv  Detail & Related papers  (2023-05-29T13:51:41Z)
• An End-to-End Cascaded Image Deraining and Object Detection Neural Network [13.314467453715517]
  In this paper, we explore the combination of the low-level vision task with the high-level vision task. We propose an end-to-end object detection network for reducing the impact of rainfall. Our network surpasses the state-of-the-art with a significant improvement in metrics.
  arXiv  Detail & Related papers  (2022-02-23T02:48:34Z)
• Online-updated High-order Collaborative Networks for Single Image Deraining [51.22694467126883]
  Single image deraining is an important task for some downstream artificial intelligence applications such as video surveillance and self-driving systems. We propose a high-order collaborative network with multi-scale compact constraints and a bidirectional scale-content similarity mining module. Our proposed method performs favorably against eleven state-of-the-art methods on five public synthetic and one real-world dataset.
  arXiv  Detail & Related papers  (2022-02-14T09:09:08Z)
• Task-Driven Deep Image Enhancement Network for Autonomous Driving in Bad Weather [5.416049433853457]
  In bad weather, visual perception is greatly affected by several degrading effects. We introduce a new task-driven training strategy to guide the high-level task model suitable for both high-quality restoration of images and highly accurate perception. Experiment results demonstrate that the proposed method improves the performance among lane and 2D object detection, and depth estimation largely under adverse weather.
  arXiv  Detail & Related papers  (2021-10-14T08:03:33Z)
• Beyond Monocular Deraining: Parallel Stereo Deraining Network Via Semantic Prior [103.49307603952144]
  Most existing de-rain algorithms use only one single input image and aim to recover a clean image. We present a Paired Rain Removal Network (PRRNet), which exploits both stereo images and semantic information. Experiments on both monocular and the newly proposed stereo rainy datasets demonstrate that the proposed method achieves the state-of-the-art performance.
  arXiv  Detail & Related papers  (2021-05-09T04:15:10Z)
• Stereo Matching by Self-supervision of Multiscopic Vision [65.38359887232025]
  We propose a new self-supervised framework for stereo matching utilizing multiple images captured at aligned camera positions. A cross photometric loss, an uncertainty-aware mutual-supervision loss, and a new smoothness loss are introduced to optimize the network. Our model obtains better disparity maps than previous unsupervised methods on the KITTI dataset.
  arXiv  Detail & Related papers  (2021-04-09T02:58:59Z)
• Distilled Semantics for Comprehensive Scene Understanding from Videos [53.49501208503774]
  In this paper, we take an additional step toward holistic scene understanding with monocular cameras by learning depth and motion alongside with semantics. We address the three tasks jointly by a novel training protocol based on knowledge distillation and self-supervision. We show that it yields state-of-the-art results for monocular depth estimation, optical flow and motion segmentation.
  arXiv  Detail & Related papers  (2020-03-31T08:52:13Z)
• Physical Model Guided Deep Image Deraining [10.14977592107907]
  Single image deraining is an urgent task because the degraded rainy image makes many computer vision systems fail to work. We propose a novel network based on physical model guided learning for single image deraining.
  arXiv  Detail & Related papers  (2020-03-30T07:08:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.