ESTANet: Efficient Online Error Detection in Procedural Videos via Prediction Inconsistency

• URL: http://arxiv.org/abs/2606.25317v1
• Date: Wed, 24 Jun 2026 02:29:57 GMT
• Title: ESTANet: Efficient Online Error Detection in Procedural Videos via Prediction Inconsistency
• Abstract summary: We study real-time online error detection in procedural videos from a simple but overlooked perspective.<n>We observe that action detectors naturally exhibit different prediction characteristics depending on their sensitivity to input dynamics and temporal context.<n>We propose ESTANet, a lightweight framework that detects errors by exploiting inconsistencies among action predictions.
• Score: 10.854541165722814
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: An efficient and accurate system for detecting errors in procedural tasks is crucial for supporting human needs in daily life, as it can provide instant notifications and guide people to correct mistakes. In this work, we study real-time online error detection in procedural videos from a simple but overlooked perspective: the prediction behavior of action detectors themselves. Instead of designing complex architectures or specialized supervision, we observe that action detectors naturally exhibit different prediction characteristics depending on their sensitivity to input dynamics and temporal context. We therefore propose ESTANet (Error-Sensitive and Temporally-vArying Network), a lightweight framework that detects errors by exploiting inconsistencies among action predictions produced by a small set of action detectors. We construct standard and error-sensitive action detectors that behave similarly on correct executions but respond differently when errors occur. Meanwhile, detectors operating with different temporal contexts further amplify prediction inconsistencies when the procedure deviates from the intended sequence. During inference, we detect errors by aggregating mismatches between standard and error-sensitive predictions through majority voting to flag frames that contain errors. Extensive experiments on EgoPER, Assembly-101-O, and EPIC-Tent-O demonstrate that ESTANet achieves state-of-the-art performance in online error detection while maintaining real-time efficiency with a lightweight architecture. Our results highlight that leveraging the intrinsic properties of action detectors can yield a powerful and practical solution for online error detection without increasing architectural design complexity.

Related papers

• Towards Automated Error Discovery: A Study in Conversational AI [48.735443116662026]
  We introduce Automated Error Discovery, a framework for detecting and defining errors in conversational AI.<n>We also propose SEEED (Soft Clustering Extended-Based Error Detection), as an encoder-based approach to its implementation.
  arXiv  Detail & Related papers  (2025-09-13T14:53:22Z)
• Technical Report for Egocentric Mistake Detection for the HoloAssist Challenge [5.257305312436567]
  We introduce an online mistake detection framework that handles both procedural and execution errors.<n>Upon detecting an error, we use a large language model (LLM) to generate explanatory feedback.<n>Experiments on the HoloAssist benchmark confirm the effectiveness of our approach.
  arXiv  Detail & Related papers  (2025-06-06T15:39:09Z)
• Modeling Multiple Normal Action Representations for Error Detection in Procedural Tasks [31.6874866836856]
  We propose an Adaptive Multiple Normal Action Representation (AMNAR) framework for error detection in procedural activities.<n>AMNAR predicts all valid next actions and reconstructs their corresponding normal action representations, which are compared against the ongoing action to detect errors.
  arXiv  Detail & Related papers  (2025-03-28T13:16:02Z)
• TI-PREGO: Chain of Thought and In-Context Learning for Online Mistake Detection in PRocedural EGOcentric Videos [48.126793563151715]
  No technique effectively detects open-set procedural mistakes online.<n>One branch continuously performs step recognition from the input egocentric video.<n>The other anticipates future steps based on the recognition module's output.
  arXiv  Detail & Related papers  (2024-11-04T20:03:06Z)
• PREGO: online mistake detection in PRocedural EGOcentric videos [49.72812518471056]
  We propose PREGO, the first online one-class classification model for mistake detection in egocentric videos. PREGO is based on an online action recognition component to model the current action, and a symbolic reasoning module to predict the next actions. We evaluate PREGO on two procedural egocentric video datasets, Assembly101 and Epic-tent, which we adapt for online benchmarking of procedural mistake detection.
  arXiv  Detail & Related papers  (2024-04-02T13:27:28Z)
• Detection Latencies of Anomaly Detectors: An Overlooked Perspective ? [1.8492669447784602]
  In this paper, we argue the relevance of measuring the temporal latency of attacks and errors. We propose an evaluation approach for detectors to ensure a pragmatic trade-off between correct and in-time detection.
  arXiv  Detail & Related papers  (2024-02-14T10:52:39Z)
• Cal-DETR: Calibrated Detection Transformer [67.75361289429013]
  We propose a mechanism for calibrated detection transformers (Cal-DETR), particularly for Deformable-DETR, UP-DETR and DINO. We develop an uncertainty-guided logit modulation mechanism that leverages the uncertainty to modulate the class logits. Results corroborate the effectiveness of Cal-DETR against the competing train-time methods in calibrating both in-domain and out-domain detections.
  arXiv  Detail & Related papers  (2023-11-06T22:13:10Z)
• A Review of Uncertainty Calibration in Pretrained Object Detectors [5.440028715314566]
  We investigate the uncertainty calibration properties of different pretrained object detection architectures in a multi-class setting. We propose a framework to ensure a fair, unbiased, and repeatable evaluation. We deliver novel insights into why poor detector calibration emerges.
  arXiv  Detail & Related papers  (2022-10-06T14:06:36Z)
• Tracking the risk of a deployed model and detecting harmful distribution shifts [105.27463615756733]
  In practice, it may make sense to ignore benign shifts, under which the performance of a deployed model does not degrade substantially. We argue that a sensible method for firing off a warning has to both (a) detect harmful shifts while ignoring benign ones, and (b) allow continuous monitoring of model performance without increasing the false alarm rate.
  arXiv  Detail & Related papers  (2021-10-12T17:21:41Z)
• Bandit Quickest Changepoint Detection [55.855465482260165]
  Continuous monitoring of every sensor can be expensive due to resource constraints. We derive an information-theoretic lower bound on the detection delay for a general class of finitely parameterized probability distributions. We propose a computationally efficient online sensing scheme, which seamlessly balances the need for exploration of different sensing options with exploitation of querying informative actions.
  arXiv  Detail & Related papers  (2021-07-22T07:25:35Z)
• On the Robustness of Language Encoders against Grammatical Errors [66.05648604987479]
  We collect real grammatical errors from non-native speakers and conduct adversarial attacks to simulate these errors on clean text data. Results confirm that the performance of all tested models is affected but the degree of impact varies.
  arXiv  Detail & Related papers  (2020-05-12T11:01:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.