Online Deep Clustering with Video Track Consistency

• URL: http://arxiv.org/abs/2206.03086v1
• Date: Tue, 7 Jun 2022 08:11:00 GMT
• Title: Online Deep Clustering with Video Track Consistency
• Authors: Alessandra Alfani, Federico Becattini, Lorenzo Seidenari, Alberto Del Bimbo
• Abstract summary: We propose an unsupervised clustering-based approach to learn visual features from video object tracks. We show that exploiting an unsupervised class-agnostic, yet noisy, track generator yields to better accuracy compared to relying on costly and precise track annotations.
• Score: 85.8868194550978
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Several unsupervised and self-supervised approaches have been developed in recent years to learn visual features from large-scale unlabeled datasets. Their main drawback however is that these methods are hardly able to recognize visual features of the same object if it is simply rotated or the perspective of the camera changes. To overcome this limitation and at the same time exploit a useful source of supervision, we take into account video object tracks. Following the intuition that two patches in a track should have similar visual representations in a learned feature space, we adopt an unsupervised clustering-based approach and constrain such representations to be labeled as the same category since they likely belong to the same object or object part. Experimental results on two downstream tasks on different datasets demonstrate the effectiveness of our Online Deep Clustering with Video Track Consistency (ODCT) approach compared to prior work, which did not leverage temporal information. In addition we show that exploiting an unsupervised class-agnostic, yet noisy, track generator yields to better accuracy compared to relying on costly and precise track annotations.

Related papers

• QDTrack: Quasi-Dense Similarity Learning for Appearance-Only Multiple Object Tracking [73.52284039530261]
  We present Quasi-Dense Similarity Learning, which densely samples hundreds of object regions on a pair of images for contrastive learning. We find that the resulting distinctive feature space admits a simple nearest neighbor search at inference time for object association. We show that our similarity learning scheme is not limited to video data, but can learn effective instance similarity even from static input.
  arXiv  Detail & Related papers  (2022-10-12T15:47:36Z)
• Semi-TCL: Semi-Supervised Track Contrastive Representation Learning [40.31083437957288]
  We design a new instance-to-track matching objective to learn appearance embedding. It compares a candidate detection to the embedding of the tracks persisted in the tracker. We implement this learning objective in a unified form following the spirit of constrastive loss.
  arXiv  Detail & Related papers  (2021-07-06T05:23:30Z)
• ASCNet: Self-supervised Video Representation Learning with Appearance-Speed Consistency [62.38914747727636]
  We study self-supervised video representation learning, which is a challenging task due to 1) a lack of labels for explicit supervision and 2) unstructured and noisy visual information. Existing methods mainly use contrastive loss with video clips as the instances and learn visual representation by discriminating instances from each other. In this paper, we observe that the consistency between positive samples is the key to learn robust video representations.
  arXiv  Detail & Related papers  (2021-06-04T08:44:50Z)
• Learning to Track Instances without Video Annotations [85.9865889886669]
  We introduce a novel semi-supervised framework by learning instance tracking networks with only a labeled image dataset and unlabeled video sequences. We show that even when only trained with images, the learned feature representation is robust to instance appearance variations. In addition, we integrate this module into single-stage instance segmentation and pose estimation frameworks.
  arXiv  Detail & Related papers  (2021-04-01T06:47:41Z)
• Learning to Track with Object Permanence [61.36492084090744]
  We introduce an end-to-end trainable approach for joint object detection and tracking. Our model, trained jointly on synthetic and real data, outperforms the state of the art on KITTI, and MOT17 datasets.
  arXiv  Detail & Related papers  (2021-03-26T04:43:04Z)
• Self-supervised Video Object Segmentation [76.83567326586162]
  The objective of this paper is self-supervised representation learning, with the goal of solving semi-supervised video object segmentation (a.k.a. dense tracking) We make the following contributions: (i) we propose to improve the existing self-supervised approach, with a simple, yet more effective memory mechanism for long-term correspondence matching; (ii) by augmenting the self-supervised approach with an online adaptation module, our method successfully alleviates tracker drifts caused by spatial-temporal discontinuity; (iv) we demonstrate state-of-the-art results among the self-supervised approaches on DAVIS-2017 and YouTube
  arXiv  Detail & Related papers  (2020-06-22T17:55:59Z)
• Unsupervised Multiple Person Tracking using AutoEncoder-Based Lifted Multicuts [11.72025865314187]
  We present an unsupervised multiple object tracking approach based on minimum visual features and lifted multicuts. We show that, despite being trained without using the provided annotations, our model provides competitive results on the challenging MOT Benchmark for pedestrian tracking.
  arXiv  Detail & Related papers  (2020-02-04T09:42:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.