IndustReal: A Dataset for Procedure Step Recognition Handling Execution Errors in Egocentric Videos in an Industrial-Like Setting

• URL: http://arxiv.org/abs/2310.17323v1
• Date: Thu, 26 Oct 2023 11:44:29 GMT
• Title: IndustReal: A Dataset for Procedure Step Recognition Handling Execution Errors in Egocentric Videos in an Industrial-Like Setting
• Authors: Tim J. Schoonbeek, Tim Houben, Hans Onvlee, Peter H.N. de With, Fons van der Sommen
• Abstract summary: We present the novel task of procedure step recognition (PSR) PSR focuses on recognizing the correct completion and order of procedural steps. We also present the multi-modal IndustReal dataset.
• Score: 7.561148568365396
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Although action recognition for procedural tasks has received notable attention, it has a fundamental flaw in that no measure of success for actions is provided. This limits the applicability of such systems especially within the industrial domain, since the outcome of procedural actions is often significantly more important than the mere execution. To address this limitation, we define the novel task of procedure step recognition (PSR), focusing on recognizing the correct completion and order of procedural steps. Alongside the new task, we also present the multi-modal IndustReal dataset. Unlike currently available datasets, IndustReal contains procedural errors (such as omissions) as well as execution errors. A significant part of these errors are exclusively present in the validation and test sets, making IndustReal suitable to evaluate robustness of algorithms to new, unseen mistakes. Additionally, to encourage reproducibility and allow for scalable approaches trained on synthetic data, the 3D models of all parts are publicly available. Annotations and benchmark performance are provided for action recognition and assembly state detection, as well as the new PSR task. IndustReal, along with the code and model weights, is available at: https://github.com/TimSchoonbeek/IndustReal .

Related papers

• PREGO: online mistake detection in PRocedural EGOcentric videos [49.72812518471056]
  We propose PREGO, the first online one-class classification model for mistake detection in egocentric videos. PREGO is based on an online action recognition component to model the current action, and a symbolic reasoning module to predict the next actions. We evaluate PREGO on two procedural egocentric video datasets, Assembly101 and Epic-tent, which we adapt for online benchmarking of procedural mistake detection.
  arXiv  Detail & Related papers  (2024-04-02T13:27:28Z)
• Fast and Accurate Unknown Object Instance Segmentation through Error-Informed Refinement [7.297340899783621]
  INSTA-BEER is a fast and accurate model-agnostic refinement method that enhances the performance of unknown object instance segmentation. We introduce the quad-metric boundary error, which quantifies pixel-wise true positives, true negatives, false positives, and false negatives at the boundaries of object instances. In comprehensive evaluations conducted on three widely used benchmark datasets, INSTA-BEER outperformed state-of-the-art models in both accuracy and inference time.
  arXiv  Detail & Related papers  (2023-06-28T12:01:51Z)
• Deep Learning based pipeline for anomaly detection and quality enhancement in industrial binder jetting processes [68.8204255655161]
  Anomaly detection describes methods of finding abnormal states, instances or data points that differ from a normal value space. This paper contributes to a data-centric way of approaching artificial intelligence in industrial production.
  arXiv  Detail & Related papers  (2022-09-21T08:14:34Z)
• SensorSCAN: Self-Supervised Learning and Deep Clustering for Fault Diagnosis in Chemical Processes [2.398451252047814]
  We propose SensorSCAN, a novel method for unsupervised fault detection and diagnosis. We demonstrate our model's performance on two publicly available datasets of the Tennessee Eastman Process with various faults. Our method is suitable for real-world applications where the number of faults is not known in advance.
  arXiv  Detail & Related papers  (2022-08-17T10:24:37Z)
• ReAct: Temporal Action Detection with Relational Queries [84.76646044604055]
  This work aims at advancing temporal action detection (TAD) using an encoder-decoder framework with action queries. We first propose a relational attention mechanism in the decoder, which guides the attention among queries based on their relations. Lastly, we propose to predict the localization quality of each action query at inference in order to distinguish high-quality queries.
  arXiv  Detail & Related papers  (2022-07-14T17:46:37Z)
• SVIP: Sequence VerIfication for Procedures in Videos [68.07865790764237]
  We propose a novel sequence verification task that aims to distinguish positive video pairs performing the same action sequence from negative ones with step-level transformations. Such a challenging task resides in an open-set setting without prior action detection or segmentation. We collect a scripted video dataset enumerating all kinds of step-level transformations in chemical experiments.
  arXiv  Detail & Related papers  (2021-12-13T07:03:36Z)
• What Stops Learning-based 3D Registration from Working in the Real World? [53.68326201131434]
  This work identifies the sources of 3D point cloud registration failures, analyze the reasons behind them, and propose solutions. Ultimately, this translates to a best-practice 3D registration network (BPNet), constituting the first learning-based method able to handle previously-unseen objects in real-world data. Our model generalizes to real data without any fine-tuning, reaching an accuracy of up to 67% on point clouds of unseen objects obtained with a commercial sensor.
  arXiv  Detail & Related papers  (2021-11-19T19:24:27Z)
• Evaluating Self and Semi-Supervised Methods for Remote Sensing Segmentation Tasks [4.7590051176368915]
  We evaluate recent self and semi-supervised ML techniques that leverage unlabeled data for improving downstream task performance. These methods are especially valuable for remote sensing tasks since there is easy access to unlabeled imagery and getting ground truth labels can often be expensive.
  arXiv  Detail & Related papers  (2021-11-19T07:41:14Z)
• Process Discovery for Structured Program Synthesis [70.29027202357385]
  A core task in process mining is process discovery which aims to learn an accurate process model from event log data. In this paper, we propose to use (block-) structured programs directly as target process models. We develop a novel bottom-up agglomerative approach to the discovery of such structured program process models.
  arXiv  Detail & Related papers  (2020-08-13T10:33:10Z)
• Active and Incremental Learning with Weak Supervision [7.2288756536476635]
  In this work, we describe combinations of an incremental learning scheme and methods of active learning. An object detection task is evaluated in a continuous exploration context on the PASCAL VOC dataset. We also validate a weakly supervised system based on active and incremental learning in a real-world biodiversity application.
  arXiv  Detail & Related papers  (2020-01-20T13:21:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.