Learning segmentation from point trajectories

• URL: http://arxiv.org/abs/2501.12392v1
• Date: Tue, 21 Jan 2025 18:59:53 GMT
• Title: Learning segmentation from point trajectories
• Authors: Laurynas Karazija, Iro Laina, Christian Rupprecht, Andrea Vedaldi,
• Abstract summary: We present a way to train a segmentation network using long-term point trajectories as a supervisory signal to complement optical flow. Our method outperforms the prior art on motion-based segmentation.
• Score: 79.02153797465326
• License:
• Abstract: We consider the problem of segmenting objects in videos based on their motion and no other forms of supervision. Prior work has often approached this problem by using the principle of common fate, namely the fact that the motion of points that belong to the same object is strongly correlated. However, most authors have only considered instantaneous motion from optical flow. In this work, we present a way to train a segmentation network using long-term point trajectories as a supervisory signal to complement optical flow. The key difficulty is that long-term motion, unlike instantaneous motion, is difficult to model -- any parametric approximation is unlikely to capture complex motion patterns over long periods of time. We instead draw inspiration from subspace clustering approaches, proposing a loss function that seeks to group the trajectories into low-rank matrices where the motion of object points can be approximately explained as a linear combination of other point tracks. Our method outperforms the prior art on motion-based segmentation, which shows the utility of long-term motion and the effectiveness of our formulation.

Related papers

• Instantaneous Perception of Moving Objects in 3D [86.38144604783207]
  The perception of 3D motion of surrounding traffic participants is crucial for driving safety. We propose to leverage local occupancy completion of object point clouds to densify the shape cue, and mitigate the impact of swimming artifacts. Extensive experiments demonstrate superior performance compared to standard 3D motion estimation approaches.
  arXiv  Detail & Related papers  (2024-05-05T01:07:24Z)
• Segmenting the motion components of a video: A long-term unsupervised model [5.801044612920816]
  We want to provide a coherent and stable motion segmentation over the video sequence. We propose a novel long-term optical-temporal model operating in a totally unsupervised way. We report experiments on four VOS, demonstrating competitive quantitative results.
  arXiv  Detail & Related papers  (2023-10-02T09:33:54Z)
• MotionTrack: Learning Robust Short-term and Long-term Motions for Multi-Object Tracking [56.92165669843006]
  We propose MotionTrack, which learns robust short-term and long-term motions in a unified framework to associate trajectories from a short to long range. For dense crowds, we design a novel Interaction Module to learn interaction-aware motions from short-term trajectories, which can estimate the complex movement of each target. For extreme occlusions, we build a novel Refind Module to learn reliable long-term motions from the target's history trajectory, which can link the interrupted trajectory with its corresponding detection.
  arXiv  Detail & Related papers  (2023-03-18T12:38:33Z)
• ParticleSfM: Exploiting Dense Point Trajectories for Localizing Moving Cameras in the Wild [57.37891682117178]
  We present a robust dense indirect structure-from-motion method for videos that is based on dense correspondence from pairwise optical flow. A novel neural network architecture is proposed for processing irregular point trajectory data. Experiments on MPI Sintel dataset show that our system produces significantly more accurate camera trajectories.
  arXiv  Detail & Related papers  (2022-07-19T09:19:45Z)
• Implicit Motion Handling for Video Camouflaged Object Detection [60.98467179649398]
  We propose a new video camouflaged object detection (VCOD) framework. It can exploit both short-term and long-term temporal consistency to detect camouflaged objects from video frames.
  arXiv  Detail & Related papers  (2022-03-14T17:55:41Z)
• FMODetect: Robust Detection and Trajectory Estimation of Fast Moving Objects [110.29738581961955]
  We propose the first learning-based approach for detection and trajectory estimation of fast moving objects. The proposed method first detects all fast moving objects as a truncated distance function to the trajectory. For the sharp appearance estimation, we propose an energy minimization based deblurring.
  arXiv  Detail & Related papers  (2020-12-15T11:05:34Z)
• MAT: Motion-Aware Multi-Object Tracking [9.098793914779161]
  In this paper, we propose Motion-Aware Tracker (MAT), focusing more on various motion patterns of different objects. Experiments on MOT16 and MOT17 challenging benchmarks demonstrate that our MAT approach can achieve the superior performance by a large margin.
  arXiv  Detail & Related papers  (2020-09-10T11:51:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.