Modeling Multiple Normal Action Representations for Error Detection in Procedural Tasks

• URL: http://arxiv.org/abs/2503.22405v2
• Date: Wed, 02 Apr 2025 04:50:12 GMT
• Title: Modeling Multiple Normal Action Representations for Error Detection in Procedural Tasks
• Abstract summary: We propose an Adaptive Multiple Normal Action Representation (AMNAR) framework for error detection in procedural activities.<n>AMNAR predicts all valid next actions and reconstructs their corresponding normal action representations, which are compared against the ongoing action to detect errors.
• Score: 31.6874866836856
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Error detection in procedural activities is essential for consistent and correct outcomes in AR-assisted and robotic systems. Existing methods often focus on temporal ordering errors or rely on static prototypes to represent normal actions. However, these approaches typically overlook the common scenario where multiple, distinct actions are valid following a given sequence of executed actions. This leads to two issues: (1) the model cannot effectively detect errors using static prototypes when the inference environment or action execution distribution differs from training; and (2) the model may also use the wrong prototypes to detect errors if the ongoing action label is not the same as the predicted one. To address this problem, we propose an Adaptive Multiple Normal Action Representation (AMNAR) framework. AMNAR predicts all valid next actions and reconstructs their corresponding normal action representations, which are compared against the ongoing action to detect errors. Extensive experiments demonstrate that AMNAR achieves state-of-the-art performance, highlighting the effectiveness of AMNAR and the importance of modeling multiple valid next actions in error detection. The code is available at https://github.com/iSEE-Laboratory/AMNAR.

Related papers

• Calibration is the Bottleneck: An Action-Class Diagnostic of Multi-Turn Tool-Calling [65.03665015319457]
  We propose an action-class-oriented diagnostic framework that decomposes multi-turn failures into two modes: action-class miscalibration and action-execution failure.<n>We validate it on a panel of tool-calling models across multiple multi-turn benchmarks.
  arXiv  Detail & Related papers  (2026-09-01T09:08:15Z)
• ESTANet: Efficient Online Error Detection in Procedural Videos via Prediction Inconsistency [10.854541165722814]
  We study real-time online error detection in procedural videos from a simple but overlooked perspective.<n>We observe that action detectors naturally exhibit different prediction characteristics depending on their sensitivity to input dynamics and temporal context.<n>We propose ESTANet, a lightweight framework that detects errors by exploiting inconsistencies among action predictions.
  arXiv  Detail & Related papers  (2026-06-24T02:29:57Z)
• VeriSpace: Spatially Grounded Action Verification for Vision-Language-Action Models [19.75611749501909]
  Vision-Ground-action (VLA) models have shown strong promise for robotic manipulation.<n>But their reliability at test time remains limited by one-shot action prediction.<n>We present VeriSpace, a 3D-aware verifier for test-time action selection in VLA systems.
  arXiv  Detail & Related papers  (2026-06-09T08:31:59Z)
• Learning Correct Behavior from Examples: Validating Sequential Execution in Autonomous Agents [6.9734639631942015]
  Traditional testing approaches require manual specification, exact sequence matching, or thousands of training examples.<n>We present a novel algorithm that automatically learns correct behavior from just 2-10 passing execution traces.<n>In controlled experiments, our system achieved high accuracy in detecting product bugs and false successes.
  arXiv  Detail & Related papers  (2026-05-04T21:07:56Z)
• Procedural Mistake Detection via Action Effect Modeling [10.358293338390716]
  Action Effect Modeling (AEM) is a unified framework that captures action execution and its outcomes through a probabilistic formulation.<n>AEM identifies the outcome of an action by selecting the most informative effect frame based on semantic relevance and visual quality.<n>It then extracts complementary cues from visual grounding and symbolic scene graphs, aligning them in a shared latent space to form robust effect-aware representations.
  arXiv  Detail & Related papers  (2025-12-03T05:56:17Z)
• Metacognitive Self-Correction for Multi-Agent System via Prototype-Guided Next-Execution Reconstruction [58.51530390018909]
  Large Language Model based multi-agent systems excel at collaborative problem solving but remain brittle to cascading errors.<n>We present MASC, a metacognitive framework that endows MAS with real-time, unsupervised, step-level error detection and self-correction.
  arXiv  Detail & Related papers  (2025-10-16T05:35:37Z)
• Technical Report for Egocentric Mistake Detection for the HoloAssist Challenge [5.257305312436567]
  We introduce an online mistake detection framework that handles both procedural and execution errors.<n>Upon detecting an error, we use a large language model (LLM) to generate explanatory feedback.<n>Experiments on the HoloAssist benchmark confirm the effectiveness of our approach.
  arXiv  Detail & Related papers  (2025-06-06T15:39:09Z)
• Bidirectional Decoding: Improving Action Chunking via Closed-Loop Resampling [51.38330727868982]
  Bidirectional Decoding (BID) is a test-time inference algorithm that bridges action chunking with closed-loop operations.<n>We show that BID boosts the performance of two state-of-the-art generative policies across seven simulation benchmarks and two real-world tasks.
  arXiv  Detail & Related papers  (2024-08-30T15:39:34Z)
• PREGO: online mistake detection in PRocedural EGOcentric videos [49.72812518471056]
  We propose PREGO, the first online one-class classification model for mistake detection in egocentric videos. PREGO is based on an online action recognition component to model the current action, and a symbolic reasoning module to predict the next actions. We evaluate PREGO on two procedural egocentric video datasets, Assembly101 and Epic-tent, which we adapt for online benchmarking of procedural mistake detection.
  arXiv  Detail & Related papers  (2024-04-02T13:27:28Z)
• DCdetector: Dual Attention Contrastive Representation Learning for Time Series Anomaly Detection [26.042898544127503]
  Time series anomaly detection is critical for a wide range of applications. It aims to identify deviant samples from the normal sample distribution in time series. We propose DCdetector, a multi-scale dual attention contrastive representation learning model.
  arXiv  Detail & Related papers  (2023-06-17T13:40:15Z)
• Abnormal Event Detection via Hypergraph Contrastive Learning [54.80429341415227]
  Abnormal event detection plays an important role in many real applications. In this paper, we study the unsupervised abnormal event detection problem in Attributed Heterogeneous Information Network. A novel hypergraph contrastive learning method, named AEHCL, is proposed to fully capture abnormal event patterns.
  arXiv  Detail & Related papers  (2023-04-02T08:23:20Z)
• DOAD: Decoupled One Stage Action Detection Network [77.14883592642782]
  Localizing people and recognizing their actions from videos is a challenging task towards high-level video understanding. Existing methods are mostly two-stage based, with one stage for person bounding box generation and the other stage for action recognition. We present a decoupled one-stage network dubbed DOAD, to improve the efficiency for-temporal action detection.
  arXiv  Detail & Related papers  (2023-04-01T08:06:43Z)
• Multivariate Time Series Anomaly Detection with Few Positive Samples [12.256288627540536]
  We introduce two methodologies to address the needs of this practical situation. Our proposed methods anchor on representative learning of normal operation with autoregressive (AR) model. We demonstrate effective performance in comparison with approaches from literature.
  arXiv  Detail & Related papers  (2022-07-02T00:58:52Z)
• Few-shot Action Recognition with Prototype-centered Attentive Learning [88.10852114988829]
  Prototype-centered Attentive Learning (PAL) model composed of two novel components. First, a prototype-centered contrastive learning loss is introduced to complement the conventional query-centered learning objective. Second, PAL integrates a attentive hybrid learning mechanism that can minimize the negative impacts of outliers.
  arXiv  Detail & Related papers  (2021-01-20T11:48:12Z)
• Spatio-Temporal Action Detection with Multi-Object Interaction [127.85524354900494]
  In this paper, we study the S-temporal action detection problem with multi-object interaction. We introduce a new dataset that is spatially annotated with action tubes containing multi-object interactions. We propose an end-to-endtemporal action detection model that performs both spatial and temporal regression simultaneously.
  arXiv  Detail & Related papers  (2020-04-01T00:54:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.