Video Waterdrop Removal via Spatio-Temporal Fusion in Driving Scenes

• URL: http://arxiv.org/abs/2302.05916v2
• Date: Wed, 15 Feb 2023 07:16:35 GMT
• Title: Video Waterdrop Removal via Spatio-Temporal Fusion in Driving Scenes
• Authors: Qiang Wen, Yue Wu, Qifeng Chen
• Abstract summary: The waterdrops on windshields during driving can cause severe visual obstructions, which may lead to car accidents. We propose an attention-based framework that fuses the representations from multiple frames to restore visual information occluded by waterdrops.
• Score: 53.16726447796844
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The waterdrops on windshields during driving can cause severe visual obstructions, which may lead to car accidents. Meanwhile, the waterdrops can also degrade the performance of a computer vision system in autonomous driving. To address these issues, we propose an attention-based framework that fuses the spatio-temporal representations from multiple frames to restore visual information occluded by waterdrops. Due to the lack of training data for video waterdrop removal, we propose a large-scale synthetic dataset with simulated waterdrops in complex driving scenes on rainy days. To improve the generality of our proposed method, we adopt a cross-modality training strategy that combines synthetic videos and real-world images. Extensive experiments show that our proposed method can generalize well and achieve the best waterdrop removal performance in complex real-world driving scenes.

Related papers

• Enhancing autonomous vehicle safety in rain: a data-centric approach for clear vision [0.0]
  We developed a vision model that processes live vehicle camera feeds to eliminate rain-induced visual hindrances. We employed a classic encoder-decoder architecture with skip connections and concatenation operations. The results demonstrated notable improvements in steering accuracy, underscoring the model's potential to enhance navigation safety and reliability in rainy weather conditions.
  arXiv  Detail & Related papers  (2024-12-29T20:27:12Z)
• StreetCrafter: Street View Synthesis with Controllable Video Diffusion Models [59.55232046525733]
  We introduce StreetCrafter, a controllable video diffusion model that utilizes LiDAR point cloud renderings as pixel-level conditions. In addition, the utilization of pixel-level LiDAR conditions allows us to make accurate pixel-level edits to target scenes. Our model enables flexible control over viewpoint changes, enlarging the view for satisfying rendering regions.
  arXiv  Detail & Related papers  (2024-12-17T18:58:55Z)
• DerainNeRF: 3D Scene Estimation with Adhesive Waterdrop Removal [12.099886168325012]
  We propose a method to reconstruct the clear 3D scene implicitly from multi-view images degraded by waterdrops. Our method exploits an attention network to predict the location of waterdrops and then train a Neural Radiance Fields to recover the 3D scene implicitly. By leveraging the strong scene representation capabilities of NeRF, our method can render high-quality novel-view images with waterdrops removed.
  arXiv  Detail & Related papers  (2024-03-29T06:58:57Z)
• NiteDR: Nighttime Image De-Raining with Cross-View Sensor Cooperative Learning for Dynamic Driving Scenes [49.92839157944134]
  In nighttime driving scenes, insufficient and uneven lighting shrouds the scenes in darkness, resulting degradation of image quality and visibility. We develop an image de-raining framework tailored for rainy nighttime driving scenes. It aims to remove rain artifacts, enrich scene representation, and restore useful information.
  arXiv  Detail & Related papers  (2024-02-28T09:02:33Z)
• EGVD: Event-Guided Video Deraining [57.59935209162314]
  We propose an end-to-end learning-based network to unlock the potential of the event camera for video deraining. We build a real-world dataset consisting of rainy videos and temporally synchronized event streams.
  arXiv  Detail & Related papers  (2023-09-29T13:47:53Z)
• ScatterNeRF: Seeing Through Fog with Physically-Based Inverse Neural Rendering [83.75284107397003]
  We introduce ScatterNeRF, a neural rendering method which renders scenes and decomposes the fog-free background. We propose a disentangled representation for the scattering volume and the scene objects, and learn the scene reconstruction with physics-inspired losses. We validate our method by capturing multi-view In-the-Wild data and controlled captures in a large-scale fog chamber.
  arXiv  Detail & Related papers  (2023-05-03T13:24:06Z)
• READ: Large-Scale Neural Scene Rendering for Autonomous Driving [21.144110676687667]
  A large-scale neural rendering method is proposed to synthesize the autonomous driving scene. Our model can not only synthesize realistic driving scenes but also stitch and edit driving scenes.
  arXiv  Detail & Related papers  (2022-05-11T14:02:14Z)
• SceneGen: Learning to Generate Realistic Traffic Scenes [92.98412203941912]
  We present SceneGen, a neural autoregressive model of traffic scenes that eschews the need for rules and distributions. We demonstrate SceneGen's ability to faithfully model distributions of real traffic scenes.
  arXiv  Detail & Related papers  (2021-01-16T22:51:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.