RAFT-MSF++: Temporal Geometry-Motion Feature Fusion for Self-Supervised Monocular Scene Flow

• URL: http://arxiv.org/abs/2604.19349v1
• Date: Tue, 21 Apr 2026 11:32:49 GMT
• Title: RAFT-MSF++: Temporal Geometry-Motion Feature Fusion for Self-Supervised Monocular Scene Flow
• Abstract summary: Monocular scene flow estimation aims to recover dense 3D motion from image sequences.<n>We propose RAFT-MSF++, a self-supervised multi-frame framework that fuses temporal features to jointly estimate depth and scene flow.<n>Experiments show that RAFT-MSF++ achieves 24.14% SF-all on the KITTI Scene Flow benchmark, with a 30.99% improvement over the baseline and better robustness in occluded regions.
• Score: 51.43025173196566
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Monocular scene flow estimation aims to recover dense 3D motion from image sequences, yet most existing methods are limited to two-frame inputs, restricting temporal modeling and robustness to occlusions. We propose RAFT-MSF++, a self-supervised multi-frame framework that recurrently fuses temporal features to jointly estimate depth and scene flow. Central to our approach is the Geometry-Motion Feature (GMF), which compactly encodes coupled motion and geometry cues and is iteratively updated for effective temporal reasoning. To ensure the robustness of this temporal fusion against occlusions, we incorporate relative positional attention to inject spatial priors and an occlusion regularization module to propagate reliable motion from visible regions. These components enable the GMF to effectively propagate information even in ambiguous areas. Extensive experiments show that RAFT-MSF++ achieves 24.14% SF-all on the KITTI Scene Flow benchmark, with a 30.99% improvement over the baseline and better robustness in occluded regions. The code is available at https://github.com/sunzunyi/RAFT-MSF-PlusPlus.

Related papers

• Making Every Step Count: Spatio-Temporal Information Allocation for Imaging Inverse Problems [63.904793414062816]
  We introduce two complementary and training-free components, i.e., Spectrum-based Scheduling (SAS) and Measurement-Prioritized Attention (MPA)<n>SAS distributes the available NFEs over flow time according to the spectrum and logSNR geometry, thus better balancing semantic exploration and detail refinement.<n>MPA guides information toward weakly constrained regions, thereby enhancing semantic and structural fidelity.
  arXiv  Detail & Related papers  (2026-08-12T07:39:22Z)
• ScanFocus: A Coarse-to-Fine Framework for Spatio-Temporal Video Grounding [23.883675818190188]
  S-Temporal Video Grounding aims to retrieve the visual trajectory of a specific object from a video stream as described by a natural language expression.<n>Most advanced methods struggle to balance global context modeling with precise boundary localization.<n>We present a novel coarse-to-fine framework that decouples the STVG task into a global-temporal scan and a local boundary focus.
  arXiv  Detail & Related papers  (2026-07-15T03:50:22Z)
• Temporal-Guided Visual Foundation Models for Event-Based Vision [40.997738547677066]
  Event cameras offer unique advantages for vision tasks in challenging environments.<n>The potential of leveraging modern Visual Foundation Models pretrained on image data remains under-explored for event-based vision.<n>We propose TemporalGuided-FM (TGVFM), a novel framework that integrates Visual Foundation Models with our temporal context fusion.
  arXiv  Detail & Related papers  (2025-11-09T05:45:25Z)
• Rethinking Temporal Fusion with a Unified Gradient Descent View for 3D Semantic Occupancy Prediction [62.69089767730514]
  We present GDFusion, a temporal fusion method for vision-based 3D semantic occupancy prediction (VisionOcc)<n>It opens up the underexplored aspects of temporal fusion within the VisionOcc framework, focusing on both temporal cues and fusion strategies.
  arXiv  Detail & Related papers  (2025-04-17T14:05:33Z)
• FUSE: Label-Free Image-Event Joint Monocular Depth Estimation via Frequency-Decoupled Alignment and Degradation-Robust Fusion [92.4205087439928]
  Image-event joint depth estimation methods leverage complementary modalities for robust perception, yet face challenges in generalizability.<n>We propose the Self-supervised Transfer (PST) and the FrequencyDe-coupled Fusion module (FreDF)<n>PST establishes cross-modal knowledge transfer through latent space alignment with image foundation models, effectively mitigating data scarcity.<n>FreDF explicitly decouples high-frequency edge features from low-frequency structural components, resolving modality-specific frequency mismatches.<n>This combined approach enables FUSE to construct a universal image-event that only requires lightweight decoder adaptation for target datasets.
  arXiv  Detail & Related papers  (2025-03-25T15:04:53Z)
• Learning Temporal 3D Semantic Scene Completion via Optical Flow Guidance [42.815766778681144]
  3D Semantic Scene Completion (SSC) provides comprehensive scene geometry and semantics for autonomous driving perception.<n>Existing SSC methods are limited to capturing sparse information from the current frame or naively stacking multi-frame temporal features.<n>We propose a novel temporal SSC method FlowScene: Learning Temporal 3D Semantic Scene Completion via Optical Flow Guidance.
  arXiv  Detail & Related papers  (2025-02-20T12:52:36Z)
• Leveraging Consistent Spatio-Temporal Correspondence for Robust Visual Odometry [7.517597541959445]
  We introduce S-Temporal Visual Odometry (STVO), a novel deep network architecture to enhance accuracy and consistency of multi-frame flow matching.<n>Our STVO achieves state-the-art performance on ETH3D benchmark and 38.9% on KITTI Odometry benchmark over the previous best methods.
  arXiv  Detail & Related papers  (2024-12-22T08:47:13Z)
• Motion-Aware Video Frame Interpolation [49.49668436390514]
  We introduce a Motion-Aware Video Frame Interpolation (MA-VFI) network, which directly estimates intermediate optical flow from consecutive frames. It not only extracts global semantic relationships and spatial details from input frames with different receptive fields, but also effectively reduces the required computational cost and complexity.
  arXiv  Detail & Related papers  (2024-02-05T11:00:14Z)
• SRFNet: Monocular Depth Estimation with Fine-grained Structure via Spatial Reliability-oriented Fusion of Frames and Events [5.800516204046145]
  Traditional frame-based methods suffer from performance drops due to limited dynamic range and motion blur. Recent works leverage novel event cameras to complement or guide the frame modality via frame-event feature fusion. SRFNet can estimate depth with fine-grained structure at both daytime and nighttime.
  arXiv  Detail & Related papers  (2023-09-22T12:59:39Z)
• Hierarchical Feature Alignment Network for Unsupervised Video Object Segmentation [99.70336991366403]
  We propose a concise, practical, and efficient architecture for appearance and motion feature alignment. The proposed HFAN reaches a new state-of-the-art performance on DAVIS-16, achieving 88.7 $mathcalJ&mathcalF$ Mean, i.e., a relative improvement of 3.5% over the best published result.
  arXiv  Detail & Related papers  (2022-07-18T10:10:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.