GMOS: Grounding Moving Object Segmentation in 3D Space and Time

• URL: http://arxiv.org/abs/2605.30352v1
• Date: Thu, 28 May 2026 17:59:58 GMT
• Title: GMOS: Grounding Moving Object Segmentation in 3D Space and Time
• Abstract summary: Moving Object (MOS) aims to discover, segment, and track objects that move independently of the camera.<n>We propose GMOS, a framework that operates directly on RGB video to produce 3D-aware, temporally fine-grained segmentation of multiple moving objects.<n>To support training and evaluation in this regime, we curate GMOS-2K, a dataset of 2,210 real-world videos with per-object temporal motion annotations.
• Score: 95.3020315930043
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Moving Object Segmentation (MOS) aims to discover, segment, and track objects that move independently of the camera. Current MOS methods, however, exhibit two fundamental limitations: they rely on pre-computed 2D auxiliary modalities such as optical flow or point trajectories that lack 3D geometric information, and they treat motion as a sequence-level attribute, overlooking the instantaneous motion state of each object. We address both by grounding MOS in 3D space and time, and propose GMOS, a framework that operates directly on RGB video to produce 3D-aware, temporally fine-grained segmentation of multiple moving objects, alongside a foreground--background variant GMOS-S for faster deployment. To support training and evaluation in this regime, we curate GMOS-2K, a dataset of 2,210 real-world videos with per-object temporal motion annotations drawn from five established Video Object Segmentation (VOS) benchmarks, and formalise MOS-I ("I" for instantaneous), a temporally fine-grained evaluation protocol with three complementary metrics. GMOS achieves state-of-the-art results across MOS, MOS-I, and Unsupervised VOS benchmarks, while running significantly faster than prior multi-object MOS methods and supporting online inference for streaming deployment.

Related papers

• MOSEv2: A More Challenging Dataset for Video Object Segmentation in Complex Scenes [131.45528437023643]
  Video object segmentation (VOS) aims to segment specified target objects throughout a video.<n>To bridge this gap, the coMplex video Object SEgmentation dataset was introduced to facilitate VOS research in complex scenes.<n>We present MOSEv2, a significantly more challenging dataset designed to further advance VOS methods under real-world conditions.
  arXiv  Detail & Related papers  (2025-08-07T17:59:27Z)
• Segment Any Motion in Videos [80.72424676419755]
  We propose a novel approach for moving object segmentation that combines long-range trajectory motion cues with DINO-based semantic features.<n>Our model employs Spatio-Temporal Trajectory Attention and Motion-Semantic Decoupled Embedding to prioritize motion while integrating semantic support.
  arXiv  Detail & Related papers  (2025-03-28T09:34:11Z)
• CV-MOS: A Cross-View Model for Motion Segmentation [13.378850442525945]
  We introduce CV-MOS, a cross-view model for moving object segmentation. We decouple spatial-temporal information by capturing the motion from BEV and RV residual maps. Our method achieved leading IoU(%) scores of 77.5% and 79.2% on the validation and test sets of the SemanticKitti dataset.
  arXiv  Detail & Related papers  (2024-08-25T09:39:26Z)
• MambaMOS: LiDAR-based 3D Moving Object Segmentation with Motion-aware State Space Model [15.418115686945056]
  LiDAR-based Moving Object (MOS) aims to locate and segment moving objects in point clouds of the current scan using motion information from previous scans. We propose a novel LiDAR-based 3D Moving Object with Motion-aware State Space Model, termed MambaMOS.
  arXiv  Detail & Related papers  (2024-04-19T11:17:35Z)
• MF-MOS: A Motion-Focused Model for Moving Object Segmentation [10.533968185642415]
  Moving object segmentation (MOS) provides a reliable solution for detecting traffic participants. Previous methods capture motion features from the range images directly. We propose MF-MOS, a novel motion-focused model with a dual-branch structure for LiDAR moving object segmentation.
  arXiv  Detail & Related papers  (2024-01-30T13:55:56Z)
• Delving into Motion-Aware Matching for Monocular 3D Object Tracking [81.68608983602581]
  We find that the motion cue of objects along different time frames is critical in 3D multi-object tracking. We propose MoMA-M3T, a framework that mainly consists of three motion-aware components. We conduct extensive experiments on the nuScenes and KITTI datasets to demonstrate our MoMA-M3T achieves competitive performance against state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-08-22T17:53:58Z)
• Event-Free Moving Object Segmentation from Moving Ego Vehicle [88.33470650615162]
  Moving object segmentation (MOS) in dynamic scenes is an important, challenging, but under-explored research topic for autonomous driving. Most segmentation methods leverage motion cues obtained from optical flow maps. We propose to exploit event cameras for better video understanding, which provide rich motion cues without relying on optical flow.
  arXiv  Detail & Related papers  (2023-04-28T23:43:10Z)
• Unsupervised 4D LiDAR Moving Object Segmentation in Stationary Settings with Multivariate Occupancy Time Series [62.997667081978825]
  We address the problem of unsupervised moving object segmentation (MOS) in 4D LiDAR data recorded from a stationary sensor, where no ground truth annotations are involved. We propose a novel 4D LiDAR representation based on a time series that relaxes the problem of unsupervised MOS. Experiments on stationary scenes from the Raw KITTI dataset show that our fully unsupervised approach achieves performance that is comparable to that of supervised state-of-the-art approaches.
  arXiv  Detail & Related papers  (2022-12-30T14:48:14Z)
• Semantics-Guided Moving Object Segmentation with 3D LiDAR [32.84782551737681]
  Moving object segmentation (MOS) is a task to distinguish moving objects from the surrounding static environment. We propose a semantics-guided convolutional neural network for moving object segmentation.
  arXiv  Detail & Related papers  (2022-05-06T12:59:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.