Detecting Object Tracking Failure via Sequential Hypothesis Testing

• URL: http://arxiv.org/abs/2602.12983v1
• Date: Fri, 13 Feb 2026 14:57:15 GMT
• Title: Detecting Object Tracking Failure via Sequential Hypothesis Testing
• Authors: Alejandro Monroy Muñoz, Rajeev Verma, Alexander Timans,
• Abstract summary: Real-time online object tracking in videos constitutes a core task in computer vision.<n>We propose interpreting object tracking as a sequential hypothesis test, wherein evidence for or against tracking failures is gradually accumulated over time.<n>We propose both supervised and unsupervised variants by leveraging either ground-truth or solely internal tracking information.
• Score: 80.7891291021747
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Real-time online object tracking in videos constitutes a core task in computer vision, with wide-ranging applications including video surveillance, motion capture, and robotics. Deployed tracking systems usually lack formal safety assurances to convey when tracking is reliable and when it may fail, at best relying on heuristic measures of model confidence to raise alerts. To obtain such assurances we propose interpreting object tracking as a sequential hypothesis test, wherein evidence for or against tracking failures is gradually accumulated over time. Leveraging recent advancements in the field, our sequential test (formalized as an e-process) quickly identifies when tracking failures set in whilst provably containing false alerts at a desired rate, and thus limiting potentially costly re-calibration or intervention steps. The approach is computationally light-weight, requires no extra training or fine-tuning, and is in principle model-agnostic. We propose both supervised and unsupervised variants by leveraging either ground-truth or solely internal tracking information, and demonstrate its effectiveness for two established tracking models across four video benchmarks. As such, sequential testing can offer a statistically grounded and efficient mechanism to incorporate safety assurances into real-time tracking systems.

Related papers

• Can We Detect Failures Without Failure Data? Uncertainty-Aware Runtime Failure Detection for Imitation Learning Policies [19.27526590452503]
  FAIL-Detect is a two-stage approach for failure detection in imitation learning-based robotic manipulation.<n>We first distill policy inputs and outputs into scalar signals that correlate with policy failures and capture uncertainty.<n>Our experiments show learned signals to be mostly consistently effective, particularly when using our novel flow-based density estimator.
  arXiv  Detail & Related papers  (2025-03-11T15:47:12Z)
• Unpacking Failure Modes of Generative Policies: Runtime Monitoring of Consistency and Progress [31.952925824381325]
  We propose a runtime monitoring framework that splits the detection of failures into two complementary categories. We use Vision Language Models (VLMs) to detect when the policy confidently and consistently takes actions that do not solve the task. By unifying temporal consistency detection and VLM runtime monitoring, Sentinel detects 18% more failures than using either of the two detectors alone.
  arXiv  Detail & Related papers  (2024-10-06T22:13:30Z)
• UTrack: Multi-Object Tracking with Uncertain Detections [37.826006378381955]
  We introduce, for the first time, a fast way to obtain the empirical predictive distribution during object detection. Our mechanism can easily be integrated into state-of-the-art trackers, enabling them to fully exploit the uncertainty in the detections. We demonstrate the effectiveness of our contribution on a variety of benchmarks, such as MOT17, MOT20, DanceTrack, and KITTI.
  arXiv  Detail & Related papers  (2024-08-30T08:34:51Z)
• RTracker: Recoverable Tracking via PN Tree Structured Memory [71.05904715104411]
  We propose a recoverable tracking framework, RTracker, that uses a tree-structured memory to dynamically associate a tracker and a detector to enable self-recovery. Specifically, we propose a Positive-Negative Tree-structured memory to chronologically store and maintain positive and negative target samples. Our core idea is to use the support samples of positive and negative target categories to establish a relative distance-based criterion for a reliable assessment of target loss.
  arXiv  Detail & Related papers  (2024-03-28T08:54:40Z)
• Tracking the risk of a deployed model and detecting harmful distribution shifts [105.27463615756733]
  In practice, it may make sense to ignore benign shifts, under which the performance of a deployed model does not degrade substantially. We argue that a sensible method for firing off a warning has to both (a) detect harmful shifts while ignoring benign ones, and (b) allow continuous monitoring of model performance without increasing the false alarm rate.
  arXiv  Detail & Related papers  (2021-10-12T17:21:41Z)
• Unsupervised Deep Representation Learning for Real-Time Tracking [137.69689503237893]
  We propose an unsupervised learning method for visual tracking. The motivation of our unsupervised learning is that a robust tracker should be effective in bidirectional tracking. We build our framework on a Siamese correlation filter network, and propose a multi-frame validation scheme and a cost-sensitive loss to facilitate unsupervised learning.
  arXiv  Detail & Related papers  (2020-07-22T08:23:12Z)
• AFAT: Adaptive Failure-Aware Tracker for Robust Visual Object Tracking [46.82222972389531]
  Siamese approaches have achieved promising performance in visual object tracking recently. Siamese paradigm uses one-shot learning to model the online tracking task, which impedes online adaptation in the tracking process. We propose a failure-aware system, based on convolutional and LSTM modules in the decision stage, enabling online reporting of potential tracking failures.
  arXiv  Detail & Related papers  (2020-05-27T23:21:12Z)
• Training-free Monocular 3D Event Detection System for Traffic Surveillance [93.65240041833319]
  Existing event detection systems are mostly learning-based and have achieved convincing performance when a large amount of training data is available. In real-world scenarios, collecting sufficient labeled training data is expensive and sometimes impossible. We propose a training-free monocular 3D event detection system for traffic surveillance.
  arXiv  Detail & Related papers  (2020-02-01T04:42:57Z)
• Confidence Trigger Detection: Accelerating Real-time Tracking-by-detection Systems [1.6037469030022993]
  Confidence-Triggered Detection (CTD) is an innovative approach that strategically bypasses object detection for frames closely resembling intermediate states. CTD not only enhances tracking speed but also preserves accuracy, surpassing existing tracking algorithms. Our experiments underscore the robustness and versatility of the CTD framework, demonstrating its potential to enable real-time tracking in resource-constrained environments.
  arXiv  Detail & Related papers  (2019-02-02T01:52:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.