How to Correctly Make Mistakes: A Framework for Constructing and Benchmarking Mistake Aware Egocentric Procedural Videos

• URL: http://arxiv.org/abs/2604.15134v1
• Date: Thu, 16 Apr 2026 15:14:29 GMT
• Title: How to Correctly Make Mistakes: A Framework for Constructing and Benchmarking Mistake Aware Egocentric Procedural Videos
• Abstract summary: We present PIE-V (Psychologically Inspired Error injection for Videos), a framework for constructing and benchmarking egocentric procedural videos.<n>For benchmarking, we introduce a unified taxonomy and a human rubric with nine metrics that cover step-level and procedure-level quality.
• Score: 18.88275830423089
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reliable procedural monitoring in video requires exposure to naturally occurring human errors and the recoveries that follow. In egocentric recordings, mistakes are often partially occluded by hands and revealed through subtle object state changes, while existing procedural datasets provide limited and inconsistent mistake and correction traces. We present PIE-V (Psychologically Inspired Error injection for Videos), a framework for constructing and benchmarking mistake-aware egocentric procedural videos by augmenting clean keystep procedures with controlled, human-plausible deviations. PIE-V combines a psychology-informed error planner conditioned on procedure phase and semantic step load, a correction planner that models recovery behavior, an LLM writer that performs cascade-consistent rewrites, and an LLM judge that validates procedural coherence and repairs failures. For video segment edits, PIE-V synthesizes replacement clips with text-guided video generation and stitches them into the episode to preserve visual plausibility. Applied to 17 tasks and 50 Ego-Exo4D scenarios, PIE-V injects 102 mistakes and generates 27 recovery corrections. For benchmarking, we introduce a unified taxonomy and a human rubric with nine metrics that cover step-level and procedure-level quality, including plausibility, procedure logic with annotator confidence, state change coherence, and grounding between text and video. Using this protocol, we audit several existing resources and compare PIE-V against a freeform LLM generation baseline under the same criteria. Together, the framework and rubric support post-completion verification for egocentric procedural mistake detection and correction.

Related papers

• Post-Training VLMs for Video Mistake Detection [30.215996979058236]
  We argue that mistake detection methods should learn the general concept of a mistake, rather than overfitting to step-specific details.<n>We introduce the Mistake Detection Video Question Answering (MD-VQA) protocol and accompanying benchmark.<n>Our method uses a tailored reward function to encourage the model to identify discrepancies between an instruction and the corresponding video.
  arXiv  Detail & Related papers  (2026-08-28T14:56:58Z)
• EgoErrorVQA: Assess Egocentric Comprehension Capabilities through Procedural Errors for Ego-Agentic AI [24.612087629174656]
  EgoErrorVQA task is proposed for egocentric procedural comprehension with explicit procedural errors modeling.<n>We develop a user-friendly evaluator agent based on the Agent2Agent protocol.<n>We also introduce Ego-ADR, an Adaptive Decoupled Reasoning framework.
  arXiv  Detail & Related papers  (2026-08-25T06:58:44Z)
• MLLM-Guided Semantic Correction for Text-to-Video Generation [66.7723669725808]
  We introduce a training-free, interpretable mid-generation correction framework that integrates multimodal large language model feedback.<n>We propose two key modules: a Semantic Assessment Supervisor that generates intermediate preview frames for semantic evaluations and deviation diagnostics, and a Semantic Modification Assistant that corrects semantic drift during inference.<n>Our method improves semantic alignment, visual fidelity, and temporal consistency without modifying model parameters.
  arXiv  Detail & Related papers  (2026-08-17T12:54:10Z)
• What Semantics Survive the Connector? Diagnosing VLM-to-DiT Alignment in Video Editing [59.40855219373921]
  Flow matching based video generative models rely on prepended Vision-Language Models to handle complex, instruction-based video editing.<n>The prevailing assumption is that a connector module can seamlessly align the VLM's rich multi-modal reasoning with the original text embedding space of DiTs.<n>We propose a controlled data processing pipeline based on video composition that results in TRACE-Edit, a diagnostic dataset focusing on relation-based editing.
  arXiv  Detail & Related papers  (2026-05-20T06:42:15Z)
• IMPACT-CYCLE: A Contract-Based Multi-Agent System for Claim-Level Supervisory Correction of Long-Video Semantic Memory [73.22944697933603]
  Existing pipelines produce opaque, end-to-end outputs that expose no intermediate state for inspection.<n>We present IMPACT-Cycle, a supervisory multi-agent system that reformulates long-video understanding as a multimodal iterative claim-level maintenance.
  arXiv  Detail & Related papers  (2026-04-22T03:03:33Z)
• ViRectify: A Challenging Benchmark for Video Reasoning Correction with Multimodal Large Language Models [23.37951284612929]
  We construct a dataset of over 30K instances spanning dynamic perception, scientific reasoning, and embodied decision-making domains.<n>In ViRectify, we challenge MLLMs to perform step-wise error identification and generate rationales with key video evidence grounding.<n>In addition, we propose the trajectory evidence-driven correction framework, comprising step-wise error trajectory and reward modeling on visual evidence-grounded correction.
  arXiv  Detail & Related papers  (2025-12-01T09:05:02Z)
• Revisiting Reconstruction-based AI-generated Image Detection: A Geometric Perspective [50.83711509908479]
  We introduce the Jacobian-Spectral Lower Bound for reconstruction error from a geometric perspective.<n>We show that real images off the reconstruction manifold exhibit a non-trivial error lower bound, while generated images on the manifold have near-zero error.<n>We propose ReGap, a training-free method that computes dynamic reconstruction error by leveraging structured editing operations.
  arXiv  Detail & Related papers  (2025-10-29T03:45:03Z)
• Metacognitive Self-Correction for Multi-Agent System via Prototype-Guided Next-Execution Reconstruction [58.51530390018909]
  Large Language Model based multi-agent systems excel at collaborative problem solving but remain brittle to cascading errors.<n>We present MASC, a metacognitive framework that endows MAS with real-time, unsupervised, step-level error detection and self-correction.
  arXiv  Detail & Related papers  (2025-10-16T05:35:37Z)
• Stable Video Infinity: Infinite-Length Video Generation with Error Recycling [76.91310169118408]
  We propose Stable Video Infinity (SVI) that is able to generate infinite-length videos with high temporal consistency, plausible scene transitions, and controllable streaming storylines.<n> SVI incorporates Error-Recycling Fine-Tuning, a new type of efficient training that recycles the Diffusion Transformer's self-generated errors into supervisory prompts.<n>We evaluate SVI on three benchmarks, including consistent, creative, and conditional settings, thoroughly verifying its versatility and state-of-the-art role.
  arXiv  Detail & Related papers  (2025-10-10T09:45:46Z)
• EdiVal-Agent: An Object-Centric Framework for Automated, Fine-Grained Evaluation of Multi-Turn Editing [170.71134330650796]
  EdiVal-Agent is an object-centric evaluation framework for instruction-based image editing.<n>It is designed to assess not only standard single-turn but also multi-turn instruction-based editing with precision.<n>We build EdiVal-Bench, a benchmark covering 9 instruction types and 13 state-of-the-art editing models spanning in-context, flow-matching, and diffusion paradigms.
  arXiv  Detail & Related papers  (2025-09-16T17:45:39Z)
• PREGO: online mistake detection in PRocedural EGOcentric videos [49.72812518471056]
  We propose PREGO, the first online one-class classification model for mistake detection in egocentric videos. PREGO is based on an online action recognition component to model the current action, and a symbolic reasoning module to predict the next actions. We evaluate PREGO on two procedural egocentric video datasets, Assembly101 and Epic-tent, which we adapt for online benchmarking of procedural mistake detection.
  arXiv  Detail & Related papers  (2024-04-02T13:27:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.