ZJU ReLER Submission for EPIC-KITCHEN Challenge 2023: TREK-150 Single Object Tracking

• URL: http://arxiv.org/abs/2307.02508v2
• Date: Mon, 10 Jul 2023 09:17:01 GMT
• Title: ZJU ReLER Submission for EPIC-KITCHEN Challenge 2023: TREK-150 Single Object Tracking
• Authors: Yuanyou Xu, Jiahao Li, Zongxin Yang, Yi Yang, Yueting Zhuang
• Abstract summary: We introduce MSDeAOT, a variant of the AOT framework that incorporates transformers at multiple feature scales. MSDeAOT efficiently propagates object masks from previous frames to the current frame using two feature scales of 16 and 8. As a testament to the effectiveness of our design, we achieved the 1st place in the EPIC-KITCHENS TREK-150 Object Tracking Challenge.
• Score: 62.98078087018469
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The Associating Objects with Transformers (AOT) framework has exhibited exceptional performance in a wide range of complex scenarios for video object tracking and segmentation. In this study, we convert the bounding boxes to masks in reference frames with the help of the Segment Anything Model (SAM) and Alpha-Refine, and then propagate the masks to the current frame, transforming the task from Video Object Tracking (VOT) to video object segmentation (VOS). Furthermore, we introduce MSDeAOT, a variant of the AOT series that incorporates transformers at multiple feature scales. MSDeAOT efficiently propagates object masks from previous frames to the current frame using two feature scales of 16 and 8. As a testament to the effectiveness of our design, we achieved the 1st place in the EPIC-KITCHENS TREK-150 Object Tracking Challenge.

Related papers

• Tracking Transforming Objects: A Benchmark [2.53045657890708]
  This study collects a novel dedicated dataset for Tracking Transforming Objects, called DTTO, which contains 100 sequences, amounting to approximately 9.3K frames. We provide carefully hand-annotated bounding boxes for each frame within these sequences, making DTTO the pioneering benchmark dedicated to tracking transforming objects. We thoroughly evaluate 20 state-of-the-art trackers on the benchmark, aiming to comprehend the performance of existing methods and provide a comparison for future research on DTTO.
  arXiv  Detail & Related papers  (2024-04-28T11:24:32Z)
• 1st Place Solution for 5th LSVOS Challenge: Referring Video Object Segmentation [65.45702890457046]
  We integrate strengths of leading RVOS models to build up an effective paradigm. To improve the consistency and quality of masks, we propose Two-Stage Multi-Model Fusion strategy. Our method achieves 75.7% J&F on Ref-Youtube-VOS validation set and 70% J&F on test set, which ranks 1st place on 5th Large-scale Video Object Challenge (ICCV 2023) track 3.
  arXiv  Detail & Related papers  (2024-01-01T04:24:48Z)
• Contrastive Learning for Multi-Object Tracking with Transformers [79.61791059432558]
  We show how DETR can be turned into a MOT model by employing an instance-level contrastive loss. Our training scheme learns object appearances while preserving detection capabilities and with little overhead. Its performance surpasses the previous state-of-the-art by +2.6 mMOTA on the challenging BDD100K dataset.
  arXiv  Detail & Related papers  (2023-11-14T10:07:52Z)
• ZJU ReLER Submission for EPIC-KITCHEN Challenge 2023: Semi-Supervised Video Object Segmentation [62.98078087018469]
  We introduce MSDeAOT, a variant of the AOT framework that incorporates transformers at multiple feature scales. MSDeAOT efficiently propagates object masks from previous frames to the current frame using a feature scale with a stride of 16. We also employ GPM in a more refined feature scale with a stride of 8, leading to improved accuracy in detecting and tracking small objects.
  arXiv  Detail & Related papers  (2023-07-05T03:43:15Z)
• The Second Place Solution for The 4th Large-scale Video Object Segmentation Challenge--Track 3: Referring Video Object Segmentation [18.630453674396534]
  ReferFormer aims to segment object instances in a given video referred by a language expression in all video frames. This work proposes several tricks to boost further, including cyclical learning rates, semi-supervised approach, and test-time augmentation inference. The improved ReferFormer ranks 2nd place on CVPR2022 Referring Youtube-VOS Challenge.
  arXiv  Detail & Related papers  (2022-06-24T02:15:06Z)
• Scalable Video Object Segmentation with Identification Mechanism [125.4229430216776]
  This paper explores the challenges of achieving scalable and effective multi-object modeling for semi-supervised Video Object (VOS) We present two innovative approaches, Associating Objects with Transformers (AOT) and Associating Objects with Scalable Transformers (AOST) Our approaches surpass the state-of-the-art competitors and display exceptional efficiency and scalability consistently across all six benchmarks.
  arXiv  Detail & Related papers  (2022-03-22T03:33:27Z)
• TrackFormer: Multi-Object Tracking with Transformers [92.25832593088421]
  TrackFormer is an end-to-end multi-object tracking and segmentation model based on an encoder-decoder Transformer architecture. New track queries are spawned by the DETR object detector and embed the position of their corresponding object over time. TrackFormer achieves a seamless data association between frames in a new tracking-by-attention paradigm.
  arXiv  Detail & Related papers  (2021-01-07T18:59:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.