SciFlow: Semantic Cross Interference for Self-Supervised Optical Flow Domain Generalization
Abstract Overview
SciFlow is a self-supervised, network-agnostic training approach for optical flow domain generalization from synthetic training data to real-world and open-world domains. Its core idea is semantic cross-domain interference: during self-supervised fine-tuning, open-world images are blended into synthetic image pairs so that a student model learns from cross-domain perturbations while a teacher model provides pseudo-targets on the original synthetic pairs. The framework also uses confidence masking and an EMA-updated teacher to stabilize learning and enforce geometric consistency. The paper positions this as a simpler alternative to methods that rely on real-world flow labels, pseudo labels, or expensive rendering and reconstruction pipelines.
Novelty
The distinctive contribution is the use of semantic cross-domain interference within a teacher-student self-supervised optical flow framework, where unlabeled open-world imagery is composited onto synthetic training pairs. This creates a simple training-time mechanism for synthetic-to-real adaptation without requiring target-domain ground truth.
Results
On KITTI benchmarks, SciFlow improves both RAFT and MobileFlow over supervised baselines and over weaker self-supervised variants that omit parts of the full method. For RAFT, the full setting reaches 2.27 Fl-epe and 8.4 Fl-all, compared with 5.04 and 17.4 for the supervised baseline; for MobileFlow, it reaches 2.78 Fl-epe and 8.95 Fl-all versus 8.20 and 22.6. Qualitative examples on DAVIS and SlowFlow also indicate more robust generalization on unseen open-world videos under conditions such as low light, motion blur, and high noise.
Key Points
- SciFlow blends unlabeled open-world images into synthetic frame pairs during self-supervised fine-tuning, using a teacher-student design with EMA and confidence masking.
- The method is architecture-agnostic and is demonstrated on both a large model (RAFT) and a lightweight model (MobileFlow).
- Experiments show improved synthetic-to-real and open-world generalization without requiring ground-truth optical flow in the target domain.