Unsupervised Cumulative Domain Adaptation for Foggy Scene Optical Flow

• URL: http://arxiv.org/abs/2303.07564v2
• Date: Mon, 20 Mar 2023 13:28:36 GMT
• Title: Unsupervised Cumulative Domain Adaptation for Foggy Scene Optical Flow
• Authors: Hanyu Zhou, Yi Chang, Wending Yan, Luxin Yan
• Abstract summary: To bridge the clean-to-foggy domain gap, the existing methods typically adopt the domain adaptation to transfer the motion knowledge from clean to synthetic foggy domain. We propose a novel unsupervised cumulative domain adaptation optical flow framework: depth-association motion adaptation and correlation-alignment motion adaptation. Under this unified framework, the proposed cumulative adaptation progressively transfers knowledge from clean scenes to real foggy scenes.
• Score: 19.640250999870307
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Optical flow has achieved great success under clean scenes, but suffers from restricted performance under foggy scenes. To bridge the clean-to-foggy domain gap, the existing methods typically adopt the domain adaptation to transfer the motion knowledge from clean to synthetic foggy domain. However, these methods unexpectedly neglect the synthetic-to-real domain gap, and thus are erroneous when applied to real-world scenes. To handle the practical optical flow under real foggy scenes, in this work, we propose a novel unsupervised cumulative domain adaptation optical flow (UCDA-Flow) framework: depth-association motion adaptation and correlation-alignment motion adaptation. Specifically, we discover that depth is a key ingredient to influence the optical flow: the deeper depth, the inferior optical flow, which motivates us to design a depth-association motion adaptation module to bridge the clean-to-foggy domain gap. Moreover, we figure out that the cost volume correlation shares similar distribution of the synthetic and real foggy images, which enlightens us to devise a correlation-alignment motion adaptation module to distill motion knowledge of the synthetic foggy domain to the real foggy domain. Note that synthetic fog is designed as the intermediate domain. Under this unified framework, the proposed cumulative adaptation progressively transfers knowledge from clean scenes to real foggy scenes. Extensive experiments have been performed to verify the superiority of the proposed method.

Related papers

• Adverse Weather Optical Flow: Cumulative Homogeneous-Heterogeneous Adaptation [36.63698348549319]
  We propose a cumulative homogeneous-heterogeneous adaptation framework for real adverse weather optical flow. Specifically, for clean-degraded transfer, our key insight is that static weather possesses the depth-association homogeneous feature which does not change the intrinsic motion of the scene. For synthetic-real transfer, we figure out that cost volume correlation shares a similar statistical histogram between synthetic and real degraded domains.
  arXiv  Detail & Related papers  (2024-09-25T15:05:03Z)
• Denoising as Adaptation: Noise-Space Domain Adaptation for Image Restoration [64.84134880709625]
  We show that it is possible to perform domain adaptation via the noise space using diffusion models. In particular, by leveraging the unique property of how auxiliary conditional inputs influence the multi-step denoising process, we derive a meaningful diffusion loss. We present crucial strategies such as channel-shuffling layer and residual-swapping contrastive learning in the diffusion model.
  arXiv  Detail & Related papers  (2024-06-26T17:40:30Z)
• D2SL: Decouple Defogging and Semantic Learning for Foggy Domain-Adaptive Segmentation [0.8261182037130406]
  We propose a novel training framework, Decouple Defogging and Semantic learning, called D2SL. We introduce a domain-consistent transfer strategy to establish a connection between defogging and segmentation tasks. We design a real fog transfer strategy to improve defogging effects by fully leveraging the fog priors from real foggy images.
  arXiv  Detail & Related papers  (2024-04-07T04:55:58Z)
• Motion-Aware Video Frame Interpolation [49.49668436390514]
  We introduce a Motion-Aware Video Frame Interpolation (MA-VFI) network, which directly estimates intermediate optical flow from consecutive frames. It not only extracts global semantic relationships and spatial details from input frames with different receptive fields, but also effectively reduces the required computational cost and complexity.
  arXiv  Detail & Related papers  (2024-02-05T11:00:14Z)
• Forward Flow for Novel View Synthesis of Dynamic Scenes [97.97012116793964]
  We propose a neural radiance field (NeRF) approach for novel view synthesis of dynamic scenes using forward warping. Our method outperforms existing methods in both novel view rendering and motion modeling.
  arXiv  Detail & Related papers  (2023-09-29T16:51:06Z)
• Unsupervised Hierarchical Domain Adaptation for Adverse Weather Optical Flow [18.900658568158054]
  We propose the first unsupervised framework for adverse weather optical flow via hierarchical motion-boundary adaptation. Our key insight is that adverse weather does not change the intrinsic optical flow of the scene, but causes a significant difference for the warp error between clean and degraded images.
  arXiv  Detail & Related papers  (2023-03-24T02:17:51Z)
• Leveraging Scale-Invariance and Uncertainity with Self-Supervised Domain Adaptation for Semantic Segmentation of Foggy Scenes [4.033107207078282]
  FogAdapt is a novel approach for domain adaptation of semantic segmentation for dense foggy scenes. FogAdapt significantly outperforms the current state-of-the-art in semantic segmentation of foggy images.
  arXiv  Detail & Related papers  (2022-01-07T18:29:58Z)
• Both Style and Fog Matter: Cumulative Domain Adaptation for Semantic Foggy Scene Understanding [63.99301797430936]
  We propose a new pipeline to cumulatively adapt style, fog and the dual-factor (style and fog) Specifically, we devise a unified framework to disentangle the style factor and the fog factor separately, and then the dual-factor from images in different domains. Our method achieves the state-of-the-art performance on three benchmarks and shows generalization ability in rainy and snowy scenes.
  arXiv  Detail & Related papers  (2021-12-01T13:21:20Z)
• Domain Adaptation for Underwater Image Enhancement [51.71570701102219]
  We propose a novel Two-phase Underwater Domain Adaptation network (TUDA) to minimize the inter-domain and intra-domain gap. In the first phase, a new dual-alignment network is designed, including a translation part for enhancing realism of input images, followed by an enhancement part. In the second phase, we perform an easy-hard classification of real data according to the assessed quality of enhanced images, where a rank-based underwater quality assessment method is embedded.
  arXiv  Detail & Related papers  (2021-08-22T06:38:19Z)
• Optical Flow Estimation from a Single Motion-blurred Image [66.2061278123057]
  Motion blur in an image may have practical interests in fundamental computer vision problems. We propose a novel framework to estimate optical flow from a single motion-blurred image in an end-to-end manner.
  arXiv  Detail & Related papers  (2021-03-04T12:45:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.