Shedding Light on Blind Spots: Developing a Reference Architecture to Leverage Video Data for Process Mining

• URL: http://arxiv.org/abs/2010.11289v3
• Date: Mon, 2 May 2022 16:13:28 GMT
• Title: Shedding Light on Blind Spots: Developing a Reference Architecture to Leverage Video Data for Process Mining
• Abstract summary: We propose a reference architecture to bridge the gap between computer vision and process mining. A prototype instantiation of the proposed reference architecture is capable of automatically extracting most of the process-relevant events from unstructured video data.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Process mining is one of the most active research streams in business process management. In recent years, numerous methods have been proposed for analyzing structured process data. Yet, in many cases, it is only the digitized parts of processes that are directly captured from process-aware information systems, and manual activities often result in blind spots. While the use of video cameras to observe these activities could help to fill this gap, a standardized approach to extracting event logs from unstructured video data remains lacking. Here, we propose a reference architecture to bridge the gap between computer vision and process mining. Various evaluation activities (i.e., competing artifact analysis, prototyping, and real-world application) ensured that the proposed reference architecture allows flexible, use-case-driven, and context-specific instantiations. Our results also show that an exemplary software prototype instantiation of the proposed reference architecture is capable of automatically extracting most of the process-relevant events from unstructured video data.

Related papers

• AI-based worker guidance in assembly and disassembly operations using multimodal ego/exo-centric data capture and structured task knowledge [9.167082845109439]
  Assembly and disassembly processes rely on expert knowledge that is difficult to document, reuse, and transfer.<n>This paper presents a data-centric approach for extracting structured task knowledge from expert demonstrations using egocentric and exocentric recordings.
  arXiv  Detail & Related papers  (2026-08-23T21:41:33Z)
• Operationalizing Document AI: A Microservice Architecture for OCR and LLM Pipelines in Production [0.07083294473439815]
  We present a microservice architecture that encapsulates pipelines of multiple models for classification, optical character recognition (OCR), and large language model structured field extraction.<n>We describe our primary design decisions, including a hybrid classification, separation of GPU-bound inference from CPU-bound orchestration, use of asynchronous processing for the many IO-bound operations in the pipeline, and an independent, horizontal scaling strategy.<n>Our goal is to provide practitioners with concrete architectural patterns for building document understanding systems that work beyond the benchmark; effectively operationalizing models in production.
  arXiv  Detail & Related papers  (2026-05-12T13:07:34Z)
• All Eyes on the Workflow: Automated and Efficient Event Discovery from Video Streams [33.04989603180585]
  We present SnapLog, an approach to extract event data from videos by converting frames to feature vectors.<n>A generalized few-shot classification is then used to assign labels to the video segments, yielding labeled, timestamped sub-sequences of frames that are interpretable as events.<n>We show that our approach produces logs that accurately reflect the process in the videos.
  arXiv  Detail & Related papers  (2026-04-24T11:54:56Z)
• Integrating Domain Knowledge into Process Discovery Using Large Language Models [3.7448613209842967]
  We propose an interactive framework that incorporates domain knowledge, expressed in natural language, into the process discovery pipeline.<n>The framework coordinates interactions among the Large Language Models (LLMs), domain experts, and a set of backend services.<n>Our empirical study includes a case study based on a real-life event log with the involvement of domain experts, who assessed the usability and effectiveness of the framework.
  arXiv  Detail & Related papers  (2025-10-08T15:59:11Z)
• Multimodal Referring Segmentation: A Survey [93.24051010753817]
  Multimodal referring segmentation aims to segment target objects in visual scenes, such as images, videos, and 3D scenes, based on referring expressions in text or audio format.<n>Over the past decade, it has gained significant attention in the multimodal community, driven by advances in convolutional neural networks, transformers, and large language models.
  arXiv  Detail & Related papers  (2025-08-01T02:14:00Z)
• IPAD: Industrial Process Anomaly Detection Dataset [71.39058003212614]
  Video anomaly detection (VAD) is a challenging task aiming to recognize anomalies in video frames. We propose a new dataset, IPAD, specifically designed for VAD in industrial scenarios. This dataset covers 16 different industrial devices and contains over 6 hours of both synthetic and real-world video footage.
  arXiv  Detail & Related papers  (2024-04-23T13:38:01Z)
• Appearance-Based Refinement for Object-Centric Motion Segmentation [85.2426540999329]
  We introduce an appearance-based refinement method that leverages temporal consistency in video streams to correct inaccurate flow-based proposals. Our approach involves a sequence-level selection mechanism that identifies accurate flow-predicted masks as exemplars. Its performance is evaluated on multiple video segmentation benchmarks, including DAVIS, YouTube, SegTrackv2, and FBMS-59.
  arXiv  Detail & Related papers  (2023-12-18T18:59:51Z)
• Deep Learning Techniques for Video Instance Segmentation: A Survey [19.32547752428875]
  Video instance segmentation is an emerging computer vision research area introduced in 2019. Deep-learning techniques take a dominant role in various computer vision areas. This survey offers a multifaceted view of deep-learning schemes for video instance segmentation.
  arXiv  Detail & Related papers  (2023-10-19T00:27:30Z)
• Discovering Hierarchical Process Models: an Approach Based on Events Clustering [0.0]
  We present an algorithm for discovering hierarchical process models represented as two-level workflow nets. Unlike existing solutions, our algorithm does not impose restrictions on the process control flow and allows for iteration.
  arXiv  Detail & Related papers  (2023-03-12T11:05:40Z)
• Clustering Object-Centric Event Logs [0.36748639131154304]
  We propose a clustering-based approach to cluster similar objects in OCELs to simplify the obtained process models. Our approach reduces the complexity of the process models and generates coherent subsets of objects which help the end-users gain insights into the process.
  arXiv  Detail & Related papers  (2022-07-26T09:16:39Z)
• Tag-Based Attention Guided Bottom-Up Approach for Video Instance Segmentation [83.13610762450703]
  Video instance is a fundamental computer vision task that deals with segmenting and tracking object instances across a video sequence. We introduce a simple end-to-end train bottomable-up approach to achieve instance mask predictions at the pixel-level granularity, instead of the typical region-proposals-based approach. Our method provides competitive results on YouTube-VIS and DAVIS-19 datasets, and has minimum run-time compared to other contemporary state-of-the-art performance methods.
  arXiv  Detail & Related papers  (2022-04-22T15:32:46Z)
• A Survey on Deep Learning Technique for Video Segmentation [147.0767454918527]
  Video segmentation plays a critical role in a broad range of practical applications. Deep learning based approaches have been dedicated to video segmentation and delivered compelling performance.
  arXiv  Detail & Related papers  (2021-07-02T15:51:07Z)
• Self-supervised Video Object Segmentation by Motion Grouping [79.13206959575228]
  We develop a computer vision system able to segment objects by exploiting motion cues. We introduce a simple variant of the Transformer to segment optical flow frames into primary objects and the background. We evaluate the proposed architecture on public benchmarks (DAVIS2016, SegTrackv2, and FBMS59)
  arXiv  Detail & Related papers  (2021-04-15T17:59:32Z)
• Process Discovery for Structured Program Synthesis [70.29027202357385]
  A core task in process mining is process discovery which aims to learn an accurate process model from event log data. In this paper, we propose to use (block-) structured programs directly as target process models. We develop a novel bottom-up agglomerative approach to the discovery of such structured program process models.
  arXiv  Detail & Related papers  (2020-08-13T10:33:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.