Fast Self-Supervised depth and mask aware Association for Multi-Object Tracking

• URL: http://arxiv.org/abs/2510.09878v1
• Date: Fri, 10 Oct 2025 21:32:24 GMT
• Title: Fast Self-Supervised depth and mask aware Association for Multi-Object Tracking
• Authors: Milad Khanchi, Maria Amer, Charalambos Poullis,
• Abstract summary: Multi-object tracking (MOT) methods often rely on Intersection-over-Union (IoU) for association.<n>In this work, we use segmentation masks to capture object shapes, but we do not compute segmentation IoU. Instead, we fuse depth and mask features and pass them through a compact encoder trained self-supervised.<n>Our MOT method is the first to use the self-supervised encoder to refine segmentation masks without computing masks IoU.
• Score: 2.676349883103404
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-object tracking (MOT) methods often rely on Intersection-over-Union (IoU) for association. However, this becomes unreliable when objects are similar or occluded. Also, computing IoU for segmentation masks is computationally expensive. In this work, we use segmentation masks to capture object shapes, but we do not compute segmentation IoU. Instead, we fuse depth and mask features and pass them through a compact encoder trained self-supervised. This encoder produces stable object representations, which we use as an additional similarity cue alongside bounding box IoU and re-identification features for matching. We obtain depth maps from a zero-shot depth estimator and object masks from a promptable visual segmentation model to obtain fine-grained spatial cues. Our MOT method is the first to use the self-supervised encoder to refine segmentation masks without computing masks IoU. MOT can be divided into joint detection-ReID (JDR) and tracking-by-detection (TBD) models. The latter are computationally more efficient. Experiments of our TBD method on challenging benchmarks with non-linear motion, occlusion, and crowded scenes, such as SportsMOT and DanceTrack, show that our method outperforms the TBD state-of-the-art on most metrics, while achieving competitive performance on simpler benchmarks with linear motion, such as MOT17.

Related papers

• PD-SORT: Occlusion-Robust Multi-Object Tracking Using Pseudo-Depth Cues [8.642829333393442]
  Multi-object tracking (MOT) is a rising topic in video processing technologies and has important application value in consumer electronics.<n>Currently, tracking-by-detection (TBD) is the dominant paradigm for MOT, which performs target detection and association frame by frame.<n>We propose Pseudo-Depth SORT (PD-SORT) to enhance the association performance and achieves leading performances on DanceTrack, MOT17, and MOT20.
  arXiv  Detail & Related papers  (2025-01-20T05:50:39Z)
• Bridge the Points: Graph-based Few-shot Segment Anything Semantically [79.1519244940518]
  Recent advancements in pre-training techniques have enhanced the capabilities of vision foundation models. Recent studies extend the SAM to Few-shot Semantic segmentation (FSS) We propose a simple yet effective approach based on graph analysis.
  arXiv  Detail & Related papers  (2024-10-09T15:02:28Z)
• LAC-Net: Linear-Fusion Attention-Guided Convolutional Network for Accurate Robotic Grasping Under the Occlusion [79.22197702626542]
  This paper introduces a framework that explores amodal segmentation for robotic grasping in cluttered scenes. We propose a Linear-fusion Attention-guided Convolutional Network (LAC-Net) The results on different datasets show that our method achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2024-08-06T14:50:48Z)
• Segment Anything Meets Point Tracking [116.44931239508578]
  This paper presents a novel method for point-centric interactive video segmentation, empowered by SAM and long-term point tracking. We highlight the merits of point-based tracking through direct evaluation on the zero-shot open-world Unidentified Video Objects (UVO) benchmark. Our experiments on popular video object segmentation and multi-object segmentation tracking benchmarks, including DAVIS, YouTube-VOS, and BDD100K, suggest that a point-based segmentation tracker yields better zero-shot performance and efficient interactions.
  arXiv  Detail & Related papers  (2023-07-03T17:58:01Z)
• SparseTrack: Multi-Object Tracking by Performing Scene Decomposition based on Pseudo-Depth [84.64121608109087]
  We propose a pseudo-depth estimation method for obtaining the relative depth of targets from 2D images. Secondly, we design a depth cascading matching (DCM) algorithm, which can use the obtained depth information to convert a dense target set into multiple sparse target subsets. By integrating the pseudo-depth method and the DCM strategy into the data association process, we propose a new tracker, called SparseTrack.
  arXiv  Detail & Related papers  (2023-06-08T14:36:10Z)
• Online Multi-Object Tracking and Segmentation with GMPHD Filter and Mask-based Affinity Fusion [79.87371506464454]
  We propose a fully online multi-object tracking and segmentation (MOTS) method that uses instance segmentation results as an input. The proposed method is based on the Gaussian mixture probability hypothesis density (GMPHD) filter, a hierarchical data association (HDA), and a mask-based affinity fusion (MAF) model. In the experiments on the two popular MOTS datasets, the key modules show some improvements.
  arXiv  Detail & Related papers  (2020-08-31T21:06:22Z)
• SipMask: Spatial Information Preservation for Fast Image and Video Instance Segmentation [149.242230059447]
  We propose a fast single-stage instance segmentation method called SipMask. It preserves instance-specific spatial information by separating mask prediction of an instance to different sub-regions of a detected bounding-box. In terms of real-time capabilities, SipMask outperforms YOLACT with an absolute gain of 3.0% (mask AP) under similar settings.
  arXiv  Detail & Related papers  (2020-07-29T12:21:00Z)
• IA-MOT: Instance-Aware Multi-Object Tracking with Motion Consistency [40.354708148590696]
  "instance-aware MOT" (IA-MOT) can track multiple objects in either static or moving cameras. Our proposed method won the first place in Track 3 of the BMTT Challenge in CVPR 2020 workshops.
  arXiv  Detail & Related papers  (2020-06-24T03:53:36Z)
• Revisiting Sequence-to-Sequence Video Object Segmentation with Multi-Task Loss and Skip-Memory [4.343892430915579]
  Video Object (VOS) is an active research area of the visual domain. Current approaches lose objects in longer sequences, especially when the object is small or briefly occluded. We build upon a sequence-to-sequence approach that employs an encoder-decoder architecture together with a memory module for exploiting the sequential data.
  arXiv  Detail & Related papers  (2020-04-25T15:38:09Z)
• PointINS: Point-based Instance Segmentation [117.38579097923052]
  Mask representation in instance segmentation with Point-of-Interest (PoI) features is challenging because learning a high-dimensional mask feature for each instance requires a heavy computing burden. We propose an instance-aware convolution, which decomposes this mask representation learning task into two tractable modules. Along with instance-aware convolution, we propose PointINS, a simple and practical instance segmentation approach.
  arXiv  Detail & Related papers  (2020-03-13T08:24:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.