MistExit: Learning to Exit for Early Mistake Detection in Procedural Videos

• URL: http://arxiv.org/abs/2603.14252v1
• Date: Sun, 15 Mar 2026 07:14:49 GMT
• Title: MistExit: Learning to Exit for Early Mistake Detection in Procedural Videos
• Abstract summary: We introduce the task of early mistake detection in video.<n>The goal is to determine whether a keystep in a procedural activity is performed correctly while observing as little of the streaming video as possible.<n>We propose a method comprising a mistake detector and a reinforcement learning policy.
• Score: 53.5354731425141
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce the task of early mistake detection in video, where the goal is to determine whether a keystep in a procedural activity is performed correctly while observing as little of the streaming video as possible. To tackle this problem, we propose a method comprising a mistake detector and a reinforcement learning policy. At each timestep, the detector processes recently observed frames to estimate the keystep's correctness while anticipating future visual features, enabling reliable early mistake estimates. Meanwhile, the policy aggregates the detector outputs and visual observations over time and adaptively decides when to exit (i.e., stop processing incoming frames) while producing the final prediction. Using diverse real-world procedural video datasets, we demonstrate that our MistExit model achieves superior mistake detection accuracy while reducing the fraction of video observed compared to state-of-the-art models. Project: https://vision.cs.utexas.edu/projects/mist_exit.

Related papers

• Post-Training VLMs for Video Mistake Detection [30.215996979058236]
  We argue that mistake detection methods should learn the general concept of a mistake, rather than overfitting to step-specific details.<n>We introduce the Mistake Detection Video Question Answering (MD-VQA) protocol and accompanying benchmark.<n>Our method uses a tailored reward function to encourage the model to identify discrepancies between an instruction and the corresponding video.
  arXiv  Detail & Related papers  (2026-08-28T14:56:58Z)
• Gazing Into Missteps: Leveraging Eye-Gaze for Unsupervised Mistake Detection in Egocentric Videos of Skilled Human Activities [25.049754180292034]
  We address the challenge of unsupervised mistake detection in egocentric video through the analysis of gaze signals.<n>Based on the observation that eye movements closely follow object manipulation activities, we assess to what extent eye-gaze signals can support mistake detection.<n>Inconsistencies between predicted and observed gaze trajectories act as an indicator to identify mistakes.
  arXiv  Detail & Related papers  (2024-06-12T16:29:45Z)
• PREGO: online mistake detection in PRocedural EGOcentric videos [49.72812518471056]
  We propose PREGO, the first online one-class classification model for mistake detection in egocentric videos. PREGO is based on an online action recognition component to model the current action, and a symbolic reasoning module to predict the next actions. We evaluate PREGO on two procedural egocentric video datasets, Assembly101 and Epic-tent, which we adapt for online benchmarking of procedural mistake detection.
  arXiv  Detail & Related papers  (2024-04-02T13:27:28Z)
• SOAR: Scene-debiasing Open-set Action Recognition [81.8198917049666]
  We propose Scene-debiasing Open-set Action Recognition (SOAR), which features an adversarial scene reconstruction module and an adaptive adversarial scene classification module. The former prevents the decoder from reconstructing the video background given video features, and thus helps reduce the background information in feature learning. The latter aims to confuse scene type classification given video features, with a specific emphasis on the action foreground, and helps to learn scene-invariant information.
  arXiv  Detail & Related papers  (2023-09-03T20:20:48Z)
• Future Video Prediction from a Single Frame for Video Anomaly Detection [0.38073142980732994]
  Video anomaly detection (VAD) is an important but challenging task in computer vision. We introduce the task of future frame prediction proxy-task, as a novel proxy-task for video anomaly detection. This proxy-task alleviates the challenges of previous methods in learning longer motion patterns.
  arXiv  Detail & Related papers  (2023-08-15T14:04:50Z)
• DiffTAD: Temporal Action Detection with Proposal Denoising Diffusion [137.8749239614528]
  We propose a new formulation of temporal action detection (TAD) with denoising diffusion, DiffTAD. Taking as input random temporal proposals, it can yield action proposals accurately given an untrimmed long video.
  arXiv  Detail & Related papers  (2023-03-27T00:40:52Z)
• Real-Time Driver Monitoring Systems through Modality and View Analysis [28.18784311981388]
  Driver distractions are known to be the dominant cause of road accidents. State-of-the-art methods prioritize accuracy while ignoring latency. We propose time-effective detection models by neglecting the temporal relation between video frames.
  arXiv  Detail & Related papers  (2022-10-17T21:22:41Z)
• Multi-Contextual Predictions with Vision Transformer for Video Anomaly Detection [22.098399083491937]
  understanding of thetemporal context of a video plays a vital role in anomaly detection. We design a transformer model with three different contextual prediction streams: masked, whole and partial. By learning to predict the missing frames of consecutive normal frames, our model can effectively learn various normality patterns in the video.
  arXiv  Detail & Related papers  (2022-06-17T05:54:31Z)
• Robust Unsupervised Video Anomaly Detection by Multi-Path Frame Prediction [61.17654438176999]
  We propose a novel and robust unsupervised video anomaly detection method by frame prediction with proper design. Our proposed method obtains the frame-level AUROC score of 88.3% on the CUHK Avenue dataset.
  arXiv  Detail & Related papers  (2020-11-05T11:34:12Z)
• Uncertainty-Aware Weakly Supervised Action Detection from Untrimmed Videos [82.02074241700728]
  In this paper, we present a prohibitive-level action recognition model that is trained with only video-frame labels. Our method per person detectors have been trained on large image datasets within Multiple Instance Learning framework. We show how we can apply our method in cases where the standard Multiple Instance Learning assumption, that each bag contains at least one instance with the specified label, is invalid.
  arXiv  Detail & Related papers  (2020-07-21T10:45:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.