CLIP-FLow: Contrastive Learning by {s}emi-supervised Iterative Pseudo {l}abeling for Optical Flow Estimation

• URL: http://arxiv.org/abs/2210.14383v1
• Date: Tue, 25 Oct 2022 23:22:25 GMT
• Title: CLIP-FLow: Contrastive Learning by {s}emi-supervised Iterative Pseudo {l}abeling for Optical Flow Estimation
• Abstract summary: We propose a semi-supervised iterative pseudo-labeling framework to transfer the pretraining knowledge to the target real domain. We leverage large-scale, unlabeled real data to facilitate transfer learning with the supervision of iteratively updated pseudo-ground truth labels. Our framework can also be extended to other models, e.g. CRAFT, reducing the F1-all error from 4.79% to 4.66% on KITTI 2015 benchmark.
• Score: 31.773232370688657
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Synthetic datasets are often used to pretrain end-to-end optical flow networks, due to the lack of a large amount of labeled, real-scene data. But major drops in accuracy occur when moving from synthetic to real scenes. How do we better transfer the knowledge learned from synthetic to real domains? To this end, we propose CLIP-FLow, a semi-supervised iterative pseudo-labeling framework to transfer the pretraining knowledge to the target real domain. We leverage large-scale, unlabeled real data to facilitate transfer learning with the supervision of iteratively updated pseudo-ground truth labels, bridging the domain gap between the synthetic and the real. In addition, we propose a contrastive flow loss on reference features and the warped features by pseudo ground truth flows, to further boost the accurate matching and dampen the mismatching due to motion, occlusion, or noisy pseudo labels. We adopt RAFT as the backbone and obtain an F1-all error of 4.11\%, i.e. a 19\% error reduction from RAFT (5.10\%) and ranking 2$^{nd}$ place at submission on KITTI 2015 benchmark. Our framework can also be extended to other models, e.g. CRAFT, reducing the F1-all error from 4.79\% to 4.66\% on KITTI 2015 benchmark.

Related papers

• On the Real-World Generalisability of Optical Flow Models [11.721555317343586]
  We study how well modern optical flow estimation models generalise to real-world video.<n>We build a real-world evaluation benchmark and evaluate the real-world generalisability of a broad set of recent optical flow models.<n>Our experiments show that progress on Sintel, KITTI and Spring only weakly predicts accuracy on real-world data.
  arXiv  Detail & Related papers  (2026-07-11T20:18:23Z)
• The Label Imitation Game: Turing Test Network for Zero-Shot Pseudo-Label Pruning [13.045700151561434]
  We introduce the Turing-inspired Label Game (LIG), a framework that formalizes pseudo-label pruning as an adversarial interrogation.<n>Rather than filtering labels via isolated thresholds, we use the LIG to train a Turing Test Network (TTN), a task-agnostic "judge"<n> Experiments across four diverse datasets demonstrate the TTN's robustness.
  arXiv  Detail & Related papers  (2026-06-29T20:06:58Z)
• Provable Sparse Inversion and Token Relabel Enhanced One-shot Federated Learning with ViTs [77.42033827176806]
  One-Shot Federated Learning, where a central server learns a global model in a single communication round, has emerged as a promising paradigm.<n>We propose a novel Federated Model Inversion and Token Relabel framework, which trains the global model by fully exploiting all patches of synthetic images.
  arXiv  Detail & Related papers  (2026-05-11T15:49:08Z)
• Pseudo2Real: Task Arithmetic for Pseudo-Label Correction in Automatic Speech Recognition [61.712328155788434]
  Real-world systems encounter unseen accents and domains with limited labeled data.<n> pseudo-labeling often introduces systematic, accent-specific errors that filtering fails to fix.<n>We propose a simple parameter-space correction to correct these recurring biases without target ground truth.
  arXiv  Detail & Related papers  (2025-10-09T10:31:47Z)
• Enhancing Zero-Shot Vision Models by Label-Free Prompt Distribution Learning and Bias Correcting [55.361337202198925]
  Vision-language models, such as CLIP, have shown impressive generalization capacities when using appropriate text descriptions. We propose a label-Free prompt distribution learning and bias correction framework, dubbed as **Frolic**, which boosts zero-shot performance without the need for labeled data.
  arXiv  Detail & Related papers  (2024-10-25T04:00:45Z)
• AdaTriplet-RA: Domain Matching via Adaptive Triplet and Reinforced Attention for Unsupervised Domain Adaptation [15.905869933337101]
  Unsupervised domain adaption (UDA) is a transfer learning task where the data and annotations of the source domain are available but only have access to the unlabeled target data during training. We propose to improve the unsupervised domain adaptation task with an inter-domain sample matching scheme. We apply the widely-used and robust Triplet loss to match the inter-domain samples. To reduce the catastrophic effect of the inaccurate pseudo-labels generated during training, we propose a novel uncertainty measurement method to select reliable pseudo-labels automatically and progressively refine them.
  arXiv  Detail & Related papers  (2022-11-16T13:04:24Z)
• Federated Learning with Label Distribution Skew via Logits Calibration [26.98248192651355]
  In this paper, we investigate the label distribution skew in FL, where the distribution of labels varies across clients. We propose FedLC, which calibrates the logits before softmax cross-entropy according to the probability of occurrence of each class. Experiments on federated datasets and real-world datasets demonstrate that FedLC leads to a more accurate global model.
  arXiv  Detail & Related papers  (2022-09-01T02:56:39Z)
• Cycle Label-Consistent Networks for Unsupervised Domain Adaptation [57.29464116557734]
  Domain adaptation aims to leverage a labeled source domain to learn a classifier for the unlabeled target domain with a different distribution. We propose a simple yet efficient domain adaptation method, i.e. Cycle Label-Consistent Network (CLCN), by exploiting the cycle consistency of classification label. We demonstrate the effectiveness of our approach on MNIST-USPS-SVHN, Office-31, Office-Home and Image CLEF-DA benchmarks.
  arXiv  Detail & Related papers  (2022-05-27T13:09:08Z)
• Imposing Consistency for Optical Flow Estimation [73.53204596544472]
  Imposing consistency through proxy tasks has been shown to enhance data-driven learning. This paper introduces novel and effective consistency strategies for optical flow estimation.
  arXiv  Detail & Related papers  (2022-04-14T22:58:30Z)
• Deformation and Correspondence Aware Unsupervised Synthetic-to-Real Scene Flow Estimation for Point Clouds [43.792032657561236]
  We develop a point cloud collector and scene flow annotator for GTA-V engine to automatically obtain diverse training samples without human intervention. We propose a mean-teacher-based domain adaptation framework that leverages self-generated pseudo-labels of the target domain. Our framework achieves superior adaptation performance on six source-target dataset pairs, remarkably closing the average domain gap by 60%.
  arXiv  Detail & Related papers  (2022-03-31T09:03:23Z)
• Synergistic Network Learning and Label Correction for Noise-robust Image Classification [28.27739181560233]
  Deep Neural Networks (DNNs) tend to overfit training label noise, resulting in poorer model performance in practice. We propose a robust label correction framework combining the ideas of small loss selection and noise correction. We demonstrate our method on both synthetic and real-world datasets with different noise types and rates.
  arXiv  Detail & Related papers  (2022-02-27T23:06:31Z)
• SCARF: Self-Supervised Contrastive Learning using Random Feature Corruption [72.35532598131176]
  We propose SCARF, a technique for contrastive learning, where views are formed by corrupting a random subset of features. We show that SCARF complements existing strategies and outperforms alternatives like autoencoders.
  arXiv  Detail & Related papers  (2021-06-29T08:08:33Z)
• Optical Flow Dataset Synthesis from Unpaired Images [36.158607790844705]
  We introduce a novel method to build a training set of pseudo-real images that can be used to train optical flow in a supervised manner. Our dataset uses two unpaired frames from real data and creates pairs of frames by simulating random warps. We thus obtain the benefit of directly training on real data while having access to an exact ground truth.
  arXiv  Detail & Related papers  (2021-04-02T22:19:47Z)
• A Free Lunch for Unsupervised Domain Adaptive Object Detection without Source Data [69.091485888121]
  Unsupervised domain adaptation assumes that source and target domain data are freely available and usually trained together to reduce the domain gap. We propose a source data-free domain adaptive object detection (SFOD) framework via modeling it into a problem of learning with noisy labels.
  arXiv  Detail & Related papers  (2020-12-10T01:42:35Z)
• RAFT: Recurrent All-Pairs Field Transforms for Optical Flow [78.92562539905951]
  We introduce Recurrent All-Pairs Field Transforms (RAFT), a new deep network architecture for optical flow. RAFT extracts per-pixel features, builds multi-scale 4D correlation volumes for all pairs of pixels, and iteratively updates a flow field. RAFT achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2020-03-26T17:12:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.