Adverse Weather Optical Flow: Cumulative Homogeneous-Heterogeneous Adaptation

• URL: http://arxiv.org/abs/2409.17001v1
• Date: Wed, 25 Sep 2024 15:05:03 GMT
• Title: Adverse Weather Optical Flow: Cumulative Homogeneous-Heterogeneous Adaptation
• Authors: Hanyu Zhou, Yi Chang, Zhiwei Shi, Wending Yan, Gang Chen, Yonghong Tian, Luxin Yan,
• Abstract summary: We propose a cumulative homogeneous-heterogeneous adaptation framework for real adverse weather optical flow. Specifically, for clean-degraded transfer, our key insight is that static weather possesses the depth-association homogeneous feature which does not change the intrinsic motion of the scene. For synthetic-real transfer, we figure out that cost volume correlation shares a similar statistical histogram between synthetic and real degraded domains.
• Score: 36.63698348549319
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Optical flow has made great progress in clean scenes, while suffers degradation under adverse weather due to the violation of the brightness constancy and gradient continuity assumptions of optical flow. Typically, existing methods mainly adopt domain adaptation to transfer motion knowledge from clean to degraded domain through one-stage adaptation. However, this direct adaptation is ineffective, since there exists a large gap due to adverse weather and scene style between clean and real degraded domains. Moreover, even within the degraded domain itself, static weather (e.g., fog) and dynamic weather (e.g., rain) have different impacts on optical flow. To address above issues, we explore synthetic degraded domain as an intermediate bridge between clean and real degraded domains, and propose a cumulative homogeneous-heterogeneous adaptation framework for real adverse weather optical flow. Specifically, for clean-degraded transfer, our key insight is that static weather possesses the depth-association homogeneous feature which does not change the intrinsic motion of the scene, while dynamic weather additionally introduces the heterogeneous feature which results in a significant boundary discrepancy in warp errors between clean and degraded domains. For synthetic-real transfer, we figure out that cost volume correlation shares a similar statistical histogram between synthetic and real degraded domains, benefiting to holistically aligning the homogeneous correlation distribution for synthetic-real knowledge distillation. Under this unified framework, the proposed method can progressively and explicitly transfer knowledge from clean scenes to real adverse weather. In addition, we further collect a real adverse weather dataset with manually annotated optical flow labels and perform extensive experiments to verify the superiority of the proposed method.

Related papers

• On Exact Editing of Flow-Based Diffusion Models [97.0633397035926]
  We propose Conditioned Velocity Correction (CVC) to reformulate flow-based editing as a distribution transformation problem driven by a known source prior.<n>CVC rethinks the role of velocity in inter-distribution transformation by introducing a dual-perspective velocity conversion mechanism.<n>We show that CVC consistently achieves superior fidelity, better semantic alignment, and more reliable editing behavior across diverse tasks.
  arXiv  Detail & Related papers  (2025-12-30T06:29:20Z)
• RoSe: Robust Self-supervised Stereo Matching under Adverse Weather Conditions [58.37558408672509]
  We propose a robust self-supervised training paradigm, consisting of two key steps: robust self-supervised scene correspondence learning and adverse weather distillation.<n>Experiments demonstrate the effectiveness and versatility of our proposed solution, which outperforms existing state-of-the-art self-supervised methods.
  arXiv  Detail & Related papers  (2025-09-23T15:41:40Z)
• A Synthetic-to-Real Dehazing Method based on Domain Unification [24.81026629589734]
  Due to distribution shift, the performance of deep learning-based method for image dehazing is adversely affected when applied to real-world images.<n>In this paper, we find that such deviation in dehazing task between real and synthetic domains may come from the imperfect collection of clean data.<n>We come up with a synthetic-to-real dehazing method based on domain unification, which attempts to unify the relationship between the real and synthetic domain.
  arXiv  Detail & Related papers  (2025-09-04T11:39:22Z)
• DACA-Net: A Degradation-Aware Conditional Diffusion Network for Underwater Image Enhancement [16.719513778795367]
  Underwater images typically suffer from severe colour distortions, low visibility, and reduced structural clarity due to complex optical effects such as scattering and absorption.<n>Existing enhancement methods often struggle to adaptively handle diverse degradation conditions and fail to leverage underwater-specific physical priors effectively.<n>We propose a degradation-aware conditional diffusion model to enhance underwater images adaptively and robustly.
  arXiv  Detail & Related papers  (2025-07-30T09:16:07Z)
• Underwater Diffusion Attention Network with Contrastive Language-Image Joint Learning for Underwater Image Enhancement [0.8747606955991707]
  UDAN-CLIP is an image-to-image diffusion framework pre-trained on synthetic underwater datasets.<n>It is enhanced with a customized classifier based on vision-language model, a spatial attention module, and a novel CLIP-Diffusion loss.<n>The proposed contributions empower our UDAN-CLIP model to perform more effective underwater image enhancement.
  arXiv  Detail & Related papers  (2025-05-26T12:24:56Z)
• Semi-Supervised State-Space Model with Dynamic Stacking Filter for Real-World Video Deraining [73.5575992346396]
  We propose a dual-branch-temporal state-space model to enhance rain streak removal in video sequences.<n>To improve multi-frame feature fusion, we derive a dynamic filter stacking, which adaptively approximates statistical filters for pixel-wise feature refinement.<n>To further explore the capacity of deraining models in supporting other vision-based tasks in rainy environments, we introduce a novel real-world benchmark.
  arXiv  Detail & Related papers  (2025-05-22T15:50:00Z)
• MicroFlow: Domain-Specific Optical Flow for Ground Deformation Estimation in Seismic Events [19.621315573734265]
  In particular, we identify that deep learning models relying on explicit correlation layers struggle at estimating small displacements in real-world conditions. Our model significantly outperforms widely used semi-synthetic geophysics benchmarks and generalizes well to challenging real-world scenarios captured by both anthropogenic and high-resolution sensors.
  arXiv  Detail & Related papers  (2025-04-18T04:10:40Z)
• SSRFlow: Semantic-aware Fusion with Spatial Temporal Re-embedding for Real-world Scene Flow [6.995663556921384]
  Scene flow provides the 3D motion field of the first frame from two consecutive point clouds. We propose a novel approach called Dual Cross Attentive (DCA) for the latent fusion and alignment between two frames based semantic contexts. We leverage novel domain adaptive losses to effectively bridge the gap of motion inference from synthetic to real-world.
  arXiv  Detail & Related papers  (2024-07-31T02:28:40Z)
• Denoising as Adaptation: Noise-Space Domain Adaptation for Image Restoration [64.84134880709625]
  We show that it is possible to perform domain adaptation via the noise space using diffusion models. In particular, by leveraging the unique property of how auxiliary conditional inputs influence the multi-step denoising process, we derive a meaningful diffusion loss. We present crucial strategies such as channel-shuffling layer and residual-swapping contrastive learning in the diffusion model.
  arXiv  Detail & Related papers  (2024-06-26T17:40:30Z)
• Typhoon Intensity Prediction with Vision Transformer [51.84456610977905]
  We introduce "Typhoon Intensity Transformer" (Tint) to predict typhoon intensity accurately across space and time. Tint uses self-attention mechanisms with global receptive fields per layer. Experiments on a publicly available typhoon benchmark validate the efficacy of Tint.
  arXiv  Detail & Related papers  (2023-11-28T03:11:33Z)
• WaterFlow: Heuristic Normalizing Flow for Underwater Image Enhancement and Beyond [52.27796682972484]
  Existing underwater image enhancement methods mainly focus on image quality improvement, ignoring the effect on practice. We propose a normalizing flow for detection-driven underwater image enhancement, dubbed WaterFlow. Considering the differentiability and interpretability, we incorporate the prior into the data-driven mapping procedure.
  arXiv  Detail & Related papers  (2023-08-02T04:17:35Z)
• Unsupervised Hierarchical Domain Adaptation for Adverse Weather Optical Flow [18.900658568158054]
  We propose the first unsupervised framework for adverse weather optical flow via hierarchical motion-boundary adaptation. Our key insight is that adverse weather does not change the intrinsic optical flow of the scene, but causes a significant difference for the warp error between clean and degraded images.
  arXiv  Detail & Related papers  (2023-03-24T02:17:51Z)
• Unsupervised Cumulative Domain Adaptation for Foggy Scene Optical Flow [19.640250999870307]
  To bridge the clean-to-foggy domain gap, the existing methods typically adopt the domain adaptation to transfer the motion knowledge from clean to synthetic foggy domain. We propose a novel unsupervised cumulative domain adaptation optical flow framework: depth-association motion adaptation and correlation-alignment motion adaptation. Under this unified framework, the proposed cumulative adaptation progressively transfers knowledge from clean scenes to real foggy scenes.
  arXiv  Detail & Related papers  (2023-03-14T01:10:59Z)
• Unsupervised Deraining: Where Asymmetric Contrastive Learning Meets Self-similarity [33.188738443097336]
  In this paper, we explore the intrinsic intra-similarity within each layer and inter-exclusiveness between two layers. We propose an unsupervised non-local contrastive learning (NLCL) deraining method.
  arXiv  Detail & Related papers  (2022-11-02T02:47:20Z)
• Domain Adaptation for Underwater Image Enhancement [51.71570701102219]
  We propose a novel Two-phase Underwater Domain Adaptation network (TUDA) to minimize the inter-domain and intra-domain gap. In the first phase, a new dual-alignment network is designed, including a translation part for enhancing realism of input images, followed by an enhancement part. In the second phase, we perform an easy-hard classification of real data according to the assessed quality of enhanced images, where a rank-based underwater quality assessment method is embedded.
  arXiv  Detail & Related papers  (2021-08-22T06:38:19Z)
• Consistency Guided Scene Flow Estimation [159.24395181068218]
  CGSF is a self-supervised framework for the joint reconstruction of 3D scene structure and motion from stereo video. We show that the proposed model can reliably predict disparity and scene flow in challenging imagery. It achieves better generalization than the state-of-the-art, and adapts quickly and robustly to unseen domains.
  arXiv  Detail & Related papers  (2020-06-19T17:28:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.