UnDIVE: Generalized Underwater Video Enhancement Using Generative Priors

• URL: http://arxiv.org/abs/2411.05886v1
• Date: Fri, 08 Nov 2024 11:16:36 GMT
• Title: UnDIVE: Generalized Underwater Video Enhancement Using Generative Priors
• Abstract summary: We propose a two-stage framework for enhancing underwater videos. The first stage uses a denoising diffusion descriptive model to learn a generative prior from unlabeled data. In the second stage, this prior is incorporated into a physics-based image formulation for spatial enhancement. Our method enables real-time and computationally-efficient processing of high-resolution underwater videos at lower resolutions.
• Score: 9.438388237767105
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: With the rise of marine exploration, underwater imaging has gained significant attention as a research topic. Underwater video enhancement has become crucial for real-time computer vision tasks in marine exploration. However, most existing methods focus on enhancing individual frames and neglect video temporal dynamics, leading to visually poor enhancements. Furthermore, the lack of ground-truth references limits the use of abundant available underwater video data in many applications. To address these issues, we propose a two-stage framework for enhancing underwater videos. The first stage uses a denoising diffusion probabilistic model to learn a generative prior from unlabeled data, capturing robust and descriptive feature representations. In the second stage, this prior is incorporated into a physics-based image formulation for spatial enhancement, while also enforcing temporal consistency between video frames. Our method enables real-time and computationally-efficient processing of high-resolution underwater videos at lower resolutions, and offers efficient enhancement in the presence of diverse water-types. Extensive experiments on four datasets show that our approach generalizes well and outperforms existing enhancement methods. Our code is available at github.com/suhas-srinath/undive.

Related papers

• Advancing Visual Reliability: Color-Accurate Underwater Image Enhancement for Real-Time Underwater Missions [52.04969451806431]
  We present a real-time underwater image enhancement framework with accurate color restoration.<n>The proposed method achieves state-of-the-art performance across seven evaluation metrics.<n>Our method improves the UCIQE score by 29.7% under diverse environmental conditions.
  arXiv  Detail & Related papers  (2026-03-17T10:51:00Z)
• WaterWave: Bridging Underwater Image Enhancement into Video Streams via Wavelet-based Temporal Consistency Field [39.2537000708326]
  We show that WaterWave significantly enhances the quality of videos generated using single-image underwater enhancements.<n>Our method demonstrates high potential in downstream underwater tracking tasks, such as UOSTrack and MAT.
  arXiv  Detail & Related papers  (2025-12-05T07:34:16Z)
• NAUTILUS: A Large Multimodal Model for Underwater Scene Understanding [60.76337064425815]
  We study the underwater scene understanding methods, which aim to achieve automated underwater exploration.<n>NautData is a dataset containing 1.45 M image-text pairs supporting eight underwater scene understanding tasks.<n>We propose a plug-and-play vision feature enhancement (VFE) module, which explicitly restores clear underwater information.
  arXiv  Detail & Related papers  (2025-10-31T14:00:35Z)
• VividFace: High-Quality and Efficient One-Step Diffusion For Video Face Enhancement [51.83206132052461]
  Video Face Enhancement (VFE) seeks to reconstruct high-quality facial regions from degraded video sequences.<n>Current methods that rely on video super-resolution and generative frameworks face three fundamental challenges.<n>We propose VividFace, a novel and efficient one-step diffusion framework for video face enhancement.
  arXiv  Detail & Related papers  (2025-09-28T02:39:48Z)
• Generative Video Matting [57.186684844156595]
  Video matting has traditionally been limited by the lack of high-quality ground-truth data.<n>Most existing video matting datasets provide only human-annotated imperfect alpha and foreground annotations.<n>We introduce a novel video matting approach that can effectively leverage the rich priors from pre-trained video diffusion models.
  arXiv  Detail & Related papers  (2025-08-11T12:18:55Z)
• Semi-Supervised State-Space Model with Dynamic Stacking Filter for Real-World Video Deraining [73.5575992346396]
  We propose a dual-branch-temporal state-space model to enhance rain streak removal in video sequences.<n>To improve multi-frame feature fusion, we derive a dynamic filter stacking, which adaptively approximates statistical filters for pixel-wise feature refinement.<n>To further explore the capacity of deraining models in supporting other vision-based tasks in rainy environments, we introduce a novel real-world benchmark.
  arXiv  Detail & Related papers  (2025-05-22T15:50:00Z)
• Video Depth Anything: Consistent Depth Estimation for Super-Long Videos [60.857723250653976]
  We propose Video Depth Anything for high-quality, consistent depth estimation in super-long videos. Our model is trained on a joint dataset of video depth and unlabeled images, similar to Depth Anything V2. Our approach sets a new state-of-the-art in zero-shot video depth estimation.
  arXiv  Detail & Related papers  (2025-01-21T18:53:30Z)
• HUPE: Heuristic Underwater Perceptual Enhancement with Semantic Collaborative Learning [62.264673293638175]
  Existing underwater image enhancement methods primarily focus on improving visual quality while overlooking practical implications. We propose a invertible network for underwater perception enhancement, dubbed H, which enhances visual quality and demonstrates flexibility in handling other downstream tasks.
  arXiv  Detail & Related papers  (2024-11-27T12:37:03Z)
• Depth Any Video with Scalable Synthetic Data [98.42356740981839]
  We develop a scalable synthetic data pipeline, capturing real-time video depth data from diverse synthetic environments. We leverage the powerful priors of generative video diffusion models to handle real-world videos effectively. Our model outperforms all previous generative depth models in terms of spatial accuracy and temporal consistency.
  arXiv  Detail & Related papers  (2024-10-14T17:59:46Z)
• IDOL: Unified Dual-Modal Latent Diffusion for Human-Centric Joint Video-Depth Generation [136.5813547244979]
  We present IDOL (unIfied Dual-mOdal Latent diffusion) for high-quality human-centric joint video-depth generation. Our IDOL consists of two novel designs. First, to enable dual-modal generation and maximize the information exchange between video and depth generation. Second, to ensure a precise video-depth spatial alignment, we propose a motion consistency loss that enforces consistency between the video and depth feature motion fields.
  arXiv  Detail & Related papers  (2024-07-15T17:36:54Z)
• LU2Net: A Lightweight Network for Real-time Underwater Image Enhancement [4.353142366661057]
  Lightweight Underwater Unet (LU2Net) is a novel U-shape network designed specifically for real-time enhancement of underwater images. LU2Net is capable of providing well-enhanced underwater images at a speed 8 times faster than the current state-of-the-art underwater image enhancement method.
  arXiv  Detail & Related papers  (2024-06-21T08:33:13Z)
• WaterMono: Teacher-Guided Anomaly Masking and Enhancement Boosting for Robust Underwater Self-Supervised Monocular Depth Estimation [4.909989222186828]
  We propose WaterMono, a novel framework for depth estimation and image enhancement. It incorporates the following key measures: (1) We present a Teacher-Guided Anomaly Mask to identify dynamic regions within the images; (2) We employ depth information combined with the Underwater Image Formation Model to generate enhanced images, which in turn contribute to the depth estimation task; and (3) We utilize a rotated distillation strategy to enhance the model's rotational robustness.
  arXiv  Detail & Related papers  (2024-06-19T08:49:45Z)
• Learning Temporally Consistent Video Depth from Video Diffusion Priors [57.929828486615605]
  This work addresses the challenge of video depth estimation. We reformulate the prediction task into a conditional generation problem. This allows us to leverage the prior knowledge embedded in existing video generation models.
  arXiv  Detail & Related papers  (2024-06-03T16:20:24Z)
• UVEB: A Large-scale Benchmark and Baseline Towards Real-World Underwater Video Enhancement [17.98919244870724]
  We construct the first large-scale high-resolution underwater video enhancement benchmark (UVEB) It contains 1,308 pairs of video sequences and more than 453,000 high-resolution with 38% Ultra-High-Definition (UHD) 4K frame pairs. We also propose the first supervised underwater video enhancement method, UVE-Net. UVE-Net converts the current frame information into convolutional kernels and passes them to adjacent frames for efficient inter-frame information exchange.
  arXiv  Detail & Related papers  (2024-04-22T19:29:12Z)
• End-To-End Underwater Video Enhancement: Dataset and Model [6.153714458213646]
  Underwater video enhancement (UVE) aims to improve the visibility and frame quality of underwater videos. Existing methods primarily focus on developing image enhancement algorithms to enhance each frame independently. This study represents the first comprehensive exploration of UVE to our knowledge.
  arXiv  Detail & Related papers  (2024-03-18T06:24:46Z)
• PUGAN: Physical Model-Guided Underwater Image Enhancement Using GAN with Dual-Discriminators [120.06891448820447]
  How to obtain clear and visually pleasant images has become a common concern of people. The task of underwater image enhancement (UIE) has also emerged as the times require. In this paper, we propose a physical model-guided GAN model for UIE, referred to as PUGAN. Our PUGAN outperforms state-of-the-art methods in both qualitative and quantitative metrics.
  arXiv  Detail & Related papers  (2023-06-15T07:41:12Z)
• Space-Time Crop & Attend: Improving Cross-modal Video Representation Learning [88.71867887257274]
  We show that spatial augmentations such as cropping work well for videos too, but that previous implementations could not do this at a scale sufficient for it to work well. To address this issue, we first introduce Feature Crop, a method to simulate such augmentations much more efficiently directly in feature space. Second, we show that as opposed to naive average pooling, the use of transformer-based attention performance improves significantly.
  arXiv  Detail & Related papers  (2021-03-18T12:32:24Z)
• Exploiting Raw Images for Real-Scene Super-Resolution [105.18021110372133]
  We study the problem of real-scene single image super-resolution to bridge the gap between synthetic data and real captured images. We propose a method to generate more realistic training data by mimicking the imaging process of digital cameras. We also develop a two-branch convolutional neural network to exploit the radiance information originally-recorded in raw images.
  arXiv  Detail & Related papers  (2021-02-02T16:10:15Z)
• Perceptual underwater image enhancement with deep learning and physical priors [35.37760003463292]
  We propose two perceptual enhancement models, each of which uses a deep enhancement model with a detection perceptor. Due to the lack of training data, a hybrid underwater image synthesis model, which fuses physical priors and data-driven cues, is proposed to synthesize training data. Experimental results show the superiority of our proposed method over several state-of-the-art methods on both real-world and synthetic underwater datasets.
  arXiv  Detail & Related papers  (2020-08-21T22:11:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.