Track Any Peppers: Weakly Supervised Sweet Pepper Tracking Using VLMs

• URL: http://arxiv.org/abs/2411.06702v1
• Date: Mon, 11 Nov 2024 04:07:25 GMT
• Title: Track Any Peppers: Weakly Supervised Sweet Pepper Tracking Using VLMs
• Abstract summary: Track Any Peppers (TAP) is a weakly supervised ensemble technique for sweet peppers tracking. TAP generates pseudo-labels for sweet peppers in video sequences with minimal human intervention. Our approach achieves a HOTA score of 80.4%, MOTA of 66.1%, Recall of 74.0%, and Precision of 90.7%.
• Score: 31.303427417198137
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: In the Detection and Multi-Object Tracking of Sweet Peppers Challenge, we present Track Any Peppers (TAP) - a weakly supervised ensemble technique for sweet peppers tracking. TAP leverages the zero-shot detection capabilities of vision-language foundation models like Grounding DINO to automatically generate pseudo-labels for sweet peppers in video sequences with minimal human intervention. These pseudo-labels, refined when necessary, are used to train a YOLOv8 segmentation network. To enhance detection accuracy under challenging conditions, we incorporate pre-processing techniques such as relighting adjustments and apply depth-based filtering during post-inference. For object tracking, we integrate the Matching by Segment Anything (MASA) adapter with the BoT-SORT algorithm. Our approach achieves a HOTA score of 80.4%, MOTA of 66.1%, Recall of 74.0%, and Precision of 90.7%, demonstrating effective tracking of sweet peppers without extensive manual effort. This work highlights the potential of foundation models for efficient and accurate object detection and tracking in agricultural settings.

Related papers

• SpikingMOT: A Spike-Driven Multi-Object Tracker [75.7974381720717]
  SpikingMOT is proposed as a spike-driven tracker that adaptively models sparse trajectory dynamics with spiking neural networks (SNNs)<n>With this brain-inspired loop, SpikingMOT achieves state-of-the-art performance in extensive experiments.
  arXiv  Detail & Related papers  (2026-07-22T08:03:23Z)
• An Improved YOLOv8 Approach for Small Target Detection of Rice Spikelet Flowering in Field Environments [1.0288898584996287]
  This study proposes a rice spikelet flowering recognition method based on an improved YOLOv8 object detection model.<n>BiFPN replaces the original PANet structure to enhance feature fusion and improve multi-scale feature utilization.<n>Given the lack of publicly available datasets for rice spikelet flowering in field conditions, a high-resolution RGB camera and data augmentation techniques are used.
  arXiv  Detail & Related papers  (2025-07-28T04:01:29Z)
• FocusTrack: A Self-Adaptive Local Sampling Algorithm for Efficient Anti-UAV Tracking [17.97806906260278]
  FocusTrack is a novel framework that dynamically refines the search region and strengthens feature representations. FocusTrack surpasses global-based trackers, requiring only 30G MACs and achieving 143 fps with FocusTrack (SRA) and 44 fps with the full version.
  arXiv  Detail & Related papers  (2025-04-18T10:18:07Z)
• FeatureSORT: Essential Features for Effective Tracking [0.0]
  FeatureSORT is an online multiple object tracker that reinforces the DeepSORT baseline with a redesigned detector and additional feature cues.<n>Our modified YOLOX architecture is extended to output multiple appearance attributes, including clothing color, clothing style, and motion direction.<n>Tests demonstrate that FeatureSORT achieves state-of-the-art online performance, with MOTA scores of 79.7 on MOT16, 80.6 on MOT17, 77.9 on MOT20, and 92.2 on DanceTrack.
  arXiv  Detail & Related papers  (2024-07-05T04:37:39Z)
• RobMOT: Robust 3D Multi-Object Tracking by Observational Noise and State Estimation Drift Mitigation on LiDAR PointCloud [11.111388829965103]
  This paper addresses limitations in 3D tracking-by-detection methods, particularly in identifying legitimate trajectories. Existing methods often use threshold-based filtering for detection scores, which can fail for distant and occluded objects. We propose a novel track validity mechanism and multi-stage observational gating process, significantly reducing ghost tracks.
  arXiv  Detail & Related papers  (2024-05-19T12:49:21Z)
• PP-YOLOE-R: An Efficient Anchor-Free Rotated Object Detector [14.263912554269435]
  PP-YOLOE-R is an anchor-free rotated object detector based on PP-YOLOE. PP-YOLOE-R-l and PP-YOLOE-R-x achieve 78.14 and 78.28 mAP on DOTA 1.0 dataset with single-scale training and testing. PP-YOLOE-R-x surpasses all anchor-free methods and demonstrates competitive performance to state-of-the-art anchor-based two-stage models.
  arXiv  Detail & Related papers  (2022-11-04T11:38:30Z)
• SOMPT22: A Surveillance Oriented Multi-Pedestrian Tracking Dataset [5.962184741057505]
  We introduce SOMPT22 dataset; a new set for multi person tracking with annotated short videos captured from static cameras located on poles with 6-8 meters in height positioned for city surveillance. We analyze MOT trackers classified as one-shot and two-stage with respect to the way of use of detection and reID networks on this new dataset. The experimental results of our new dataset indicate that SOTA is still far from high efficiency, and single-shot trackers are good candidates to unify fast execution and accuracy with competitive performance.
  arXiv  Detail & Related papers  (2022-08-04T11:09:19Z)
• SparseTT: Visual Tracking with Sparse Transformers [43.1666514605021]
  Self-attention mechanism designed to model long-range dependencies is the key to the success of Transformers. In this paper, we relieve this issue with a sparse attention mechanism by focusing the most relevant information in the search regions. We introduce a double-head predictor to boost the accuracy of foreground-background classification and regression of target bounding boxes.
  arXiv  Detail & Related papers  (2022-05-08T04:00:28Z)
• VariabilityTrack:Multi-Object Tracking with Variable Speed Object Movement [1.6385815610837167]
  Multi-object tracking (MOT) aims at estimating bounding boxes and identities of objects in videos. We propose a variable speed Kalman filter algorithm based on environmental feedback and improve the matching process.
  arXiv  Detail & Related papers  (2022-03-12T12:39:41Z)
• SwinTrack: A Simple and Strong Baseline for Transformer Tracking [81.65306568735335]
  We propose a fully attentional-based Transformer tracking algorithm, Swin-Transformer Tracker (SwinTrack) SwinTrack uses Transformer for both feature extraction and feature fusion, allowing full interactions between the target object and the search region for tracking. In our thorough experiments, SwinTrack sets a new record with 0.717 SUC on LaSOT, surpassing STARK by 4.6% while still running at 45 FPS.
  arXiv  Detail & Related papers  (2021-12-02T05:56:03Z)
• Rethinking Keypoint Representations: Modeling Keypoints and Poses as Objects for Multi-Person Human Pose Estimation [79.78017059539526]
  We propose a new heatmap-free keypoint estimation method in which individual keypoints and sets of spatially related keypoints (i.e., poses) are modeled as objects within a dense single-stage anchor-based detection framework. In experiments, we observe that KAPAO is significantly faster and more accurate than previous methods, which suffer greatly from heatmap post-processing. Our large model, KAPAO-L, achieves an AP of 70.6 on the Microsoft COCO Keypoints validation set without test-time augmentation.
  arXiv  Detail & Related papers  (2021-11-16T15:36:44Z)
• Learning Target Candidate Association to Keep Track of What Not to Track [100.80610986625693]
  We propose to keep track of distractor objects in order to continue tracking the target. To tackle the problem of lacking ground-truth correspondences between distractor objects in visual tracking, we propose a training strategy that combines partial annotations with self-supervision. Our tracker sets a new state-of-the-art on six benchmarks, achieving an AUC score of 67.2% on LaSOT and a +6.1% absolute gain on the OxUvA long-term dataset.
  arXiv  Detail & Related papers  (2021-03-30T17:58:02Z)
• Tracklets Predicting Based Adaptive Graph Tracking [51.352829280902114]
  We present an accurate and end-to-end learning framework for multi-object tracking, namely textbfTPAGT. It re-extracts the features of the tracklets in the current frame based on motion predicting, which is the key to solve the problem of features inconsistent.
  arXiv  Detail & Related papers  (2020-10-18T16:16:49Z)
• Cooling-Shrinking Attack: Blinding the Tracker with Imperceptible Noises [87.53808756910452]
  A cooling-shrinking attack method is proposed to deceive state-of-the-art SiameseRPN-based trackers. Our method has good transferability and is able to deceive other top-performance trackers such as DaSiamRPN, DaSiamRPN-UpdateNet, and DiMP.
  arXiv  Detail & Related papers  (2020-03-21T07:13:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.