Leveraging Foundation Models via Knowledge Distillation in Multi-Object Tracking: Distilling DINOv2 Features to FairMOT

• URL: http://arxiv.org/abs/2407.18288v2
• Date: Mon, 5 Aug 2024 06:50:44 GMT
• Title: Leveraging Foundation Models via Knowledge Distillation in Multi-Object Tracking: Distilling DINOv2 Features to FairMOT
• Authors: Niels G. Faber, Seyed Sahand Mohammadi Ziabari, Fatemeh Karimi Nejadasl,
• Abstract summary: This work tries to leverage one such foundation model, called DINOv2, through using knowledge distillation. The results imply that although the proposed method shows improvements in certain scenarios, it does not consistently outperform the original FairMOT model.
• Score: 0.5999777817331317
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multiple Object Tracking (MOT) is a computer vision task that has been employed in a variety of sectors. Some common limitations in MOT are varying object appearances, occlusions, or crowded scenes. To address these challenges, machine learning methods have been extensively deployed, leveraging large datasets, sophisticated models, and substantial computational resources. Due to practical limitations, access to the above is not always an option. However, with the recent release of foundation models by prominent AI companies, pretrained models have been trained on vast datasets and resources using state-of-the-art methods. This work tries to leverage one such foundation model, called DINOv2, through using knowledge distillation. The proposed method uses a teacher-student architecture, where DINOv2 is the teacher and the FairMOT backbone HRNetv2 W18 is the student. The results imply that although the proposed method shows improvements in certain scenarios, it does not consistently outperform the original FairMOT model. These findings highlight the potential and limitations of applying foundation models in knowledge

Related papers

• iNeMo: Incremental Neural Mesh Models for Robust Class-Incremental Learning [22.14627083675405]
  We propose incremental neural mesh models that can be extended with new meshes over time. We demonstrate the effectiveness of our method through extensive experiments on the Pascal3D and ObjectNet3D datasets. Our work also presents the first incremental learning approach for pose estimation.
  arXiv  Detail & Related papers  (2024-07-12T13:57:49Z)
• Low-resource finetuning of foundation models beats state-of-the-art in histopathology [3.4577420145036375]
  We benchmark the most popular vision foundation models as feature extractors for histopathology data. By finetuning a foundation model on a single GPU for only two hours or three days depending on the dataset, we can match or outperform state-of-the-art feature extractors. This is a considerable shift from the current state, where only few institutions with large amounts of resources and datasets are able to train a feature extractor.
  arXiv  Detail & Related papers  (2024-01-09T18:46:59Z)
• Learn From Model Beyond Fine-Tuning: A Survey [78.80920533793595]
  Learn From Model (LFM) focuses on the research, modification, and design of foundation models (FM) based on the model interface. The study of LFM techniques can be broadly categorized into five major areas: model tuning, model distillation, model reuse, meta learning and model editing. This paper gives a comprehensive review of the current methods based on FM from the perspective of LFM.
  arXiv  Detail & Related papers  (2023-10-12T10:20:36Z)
• MinT: Boosting Generalization in Mathematical Reasoning via Multi-View Fine-Tuning [53.90744622542961]
  Reasoning in mathematical domains remains a significant challenge for small language models (LMs) We introduce a new method that exploits existing mathematical problem datasets with diverse annotation styles. Experimental results show that our strategy enables a LLaMA-7B model to outperform prior approaches.
  arXiv  Detail & Related papers  (2023-07-16T05:41:53Z)
• OVTrack: Open-Vocabulary Multiple Object Tracking [64.73379741435255]
  OVTrack is an open-vocabulary tracker capable of tracking arbitrary object classes. It sets a new state-of-the-art on the large-scale, large-vocabulary TAO benchmark.
  arXiv  Detail & Related papers  (2023-04-17T16:20:05Z)
• DINOv2: Learning Robust Visual Features without Supervision [75.42921276202522]
  This work shows that existing pretraining methods, especially self-supervised methods, can produce such features if trained on enough curated data from diverse sources. Most of the technical contributions aim at accelerating and stabilizing the training at scale. In terms of data, we propose an automatic pipeline to build a dedicated, diverse, and curated image dataset instead of uncurated data, as typically done in the self-supervised literature.
  arXiv  Detail & Related papers  (2023-04-14T15:12:19Z)
• Towards Efficient Task-Driven Model Reprogramming with Foundation Models [52.411508216448716]
  Vision foundation models exhibit impressive power, benefiting from the extremely large model capacity and broad training data. However, in practice, downstream scenarios may only support a small model due to the limited computational resources or efficiency considerations. This brings a critical challenge for the real-world application of foundation models: one has to transfer the knowledge of a foundation model to the downstream task.
  arXiv  Detail & Related papers  (2023-04-05T07:28:33Z)
• MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation [68.30497162547768]
  We propose MoEBERT, which uses a Mixture-of-Experts structure to increase model capacity and inference speed. We validate the efficiency and effectiveness of MoEBERT on natural language understanding and question answering tasks.
  arXiv  Detail & Related papers  (2022-04-15T23:19:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.