ExOAR: Expert-Guided Object and Activity Recognition from Textual Data

• URL: http://arxiv.org/abs/2512.03790v1
• Date: Wed, 03 Dec 2025 13:40:17 GMT
• Title: ExOAR: Expert-Guided Object and Activity Recognition from Textual Data
• Abstract summary: ExOAR (Expert-Guided Object and Activity Recognition) is an interactive method that combines large language models (LLMs) with human verification to identify objects and activities from textual data.<n>We show that ExOAR can effectively bridge the gap between unstructured textual data and the structured log with clear semantics needed for object-centric process analysis.
• Score: 0.3441021278275805
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Object-centric process mining requires structured data, but extracting it from unstructured text remains a challenge. We introduce ExOAR (Expert-Guided Object and Activity Recognition), an interactive method that combines large language models (LLMs) with human verification to identify objects and activities from textual data. ExOAR guides users through consecutive stages in which an LLM generates candidate object types, activities, and object instances based on contextual input, such as a user's profession, and textual data. Users review and refine these suggestions before proceeding to the next stage. Implemented as a practical tool, ExOAR is initially validated through a demonstration and then evaluated with real-world Active Window Tracking data from five users. Our results show that ExOAR can effectively bridge the gap between unstructured textual data and the structured log with clear semantics needed for object-centric process analysis, while it maintains flexibility and human oversight.

Related papers

• Relational Semantic Reasoning on 3D Scene Graphs for Open World Interactive Object Search [14.297888615008041]
  We introduce SCOUT: Scene Graph-Based Exploration with Learned Utility for Open-World Interactive Object Search.<n>SCOUT searches directly over 3D scene graphs by assigning utility scores to rooms, frontiers, and objects.<n>We present SymSearch, a scalable symbolic benchmark for evaluating semantic reasoning in interactive object search tasks.
  arXiv  Detail & Related papers  (2026-03-05T19:52:26Z)
• Named Entity Recognition of Historical Text via Large Language Model [1.5469452301122175]
  Named Entity Recognition (NER) plays a crucial role in extracting information from unstructured text.<n>Traditionally, NER is addressed using supervised machine learning approaches, which require large amounts of annotated training data.<n>In this study, we explore the feasibility of applying LLMs to NER in historical documents using zero-shot and few-shot prompting strategies.<n>Our experiments, conducted on the HIPE-2022 dataset, show that LLMs can achieve reasonably strong performance on NER tasks in this setting.
  arXiv  Detail & Related papers  (2025-08-25T14:52:11Z)
• DataScout: Automatic Data Fact Retrieval for Statement Augmentation with an LLM-Based Agent [18.424165885187968]
  DataScout is an interactive system that automatically performs reasoning and stance-based data facts retrieval.<n>The interface visualizes the retrieval tree as a mind map that eases users to intuitively steer the retrieval direction.<n>Our evaluation demonstrates that DataScout can effectively retrieve multifaceted data facts from different stances, helping users verify their statements and enhance the credibility of their stories.
  arXiv  Detail & Related papers  (2025-04-24T07:52:09Z)
• Conversational Text Extraction with Large Language Models Using Retrieval-Augmented Systems [0.20971479389679337]
  This study introduces a system leveraging Large Language Models (LLMs) to extract text from PDF documents via a conversational interface.<n>The system provides informative responses to user inquiries while highlighting relevant passages within the PDF.<n>The proposed system gives competitive ROUGE values as compared to existing state-of-the-art techniques for text extraction and summarization.
  arXiv  Detail & Related papers  (2025-01-16T19:12:25Z)
• Spatio-Temporal Context Prompting for Zero-Shot Action Detection [13.22912547389941]
  We propose a method which can effectively leverage the rich knowledge of visual-language models to perform Person-Context Interaction.<n>To address the challenge of recognizing distinct actions by multiple people at the same timestamp, we design the Interest Token Spotting mechanism.<n>Our method achieves superior results compared to previous approaches and can be further extended to multi-action videos.
  arXiv  Detail & Related papers  (2024-08-28T17:59:05Z)
• Learning to Extract Structured Entities Using Language Models [52.281701191329]
  Recent advances in machine learning have significantly impacted the field of information extraction. We reformulate the task to be entity-centric, enabling the use of diverse metrics. We contribute to the field by introducing Structured Entity Extraction and proposing the Approximate Entity Set OverlaP metric.
  arXiv  Detail & Related papers  (2024-02-06T22:15:09Z)
• Open-Vocabulary Camouflaged Object Segmentation [66.94945066779988]
  We introduce a new task, open-vocabulary camouflaged object segmentation (OVCOS) We construct a large-scale complex scene dataset (textbfOVCamo) containing 11,483 hand-selected images with fine annotations and corresponding object classes. By integrating the guidance of class semantic knowledge and the supplement of visual structure cues from the edge and depth information, the proposed method can efficiently capture camouflaged objects.
  arXiv  Detail & Related papers  (2023-11-19T06:00:39Z)
• Instruct and Extract: Instruction Tuning for On-Demand Information Extraction [86.29491354355356]
  On-Demand Information Extraction aims to fulfill the personalized demands of real-world users. We present a benchmark named InstructIE, inclusive of both automatically generated training data, as well as the human-annotated test set. Building on InstructIE, we further develop an On-Demand Information Extractor, ODIE.
  arXiv  Detail & Related papers  (2023-10-24T17:54:25Z)
• LISA: Reasoning Segmentation via Large Language Model [68.24075852136761]
  We propose a new segmentation task -- reasoning segmentation. The task is designed to output a segmentation mask given a complex and implicit query text. We present LISA: large Language Instructed Assistant, which inherits the language generation capabilities of multimodal Large Language Models.
  arXiv  Detail & Related papers  (2023-08-01T17:50:17Z)
• Layout-Aware Information Extraction for Document-Grounded Dialogue: Dataset, Method and Demonstration [75.47708732473586]
  We propose a layout-aware document-level Information Extraction dataset, LIE, to facilitate the study of extracting both structural and semantic knowledge from visually rich documents. LIE contains 62k annotations of three extraction tasks from 4,061 pages in product and official documents. Empirical results show that layout is critical for VRD-based extraction, and system demonstration also verifies that the extracted knowledge can help locate the answers that users care about.
  arXiv  Detail & Related papers  (2022-07-14T07:59:45Z)
• Detecting Ongoing Events Using Contextual Word and Sentence Embeddings [110.83289076967895]
  This paper introduces the Ongoing Event Detection (OED) task. The goal is to detect ongoing event mentions only, as opposed to historical, future, hypothetical, or other forms or events that are neither fresh nor current. Any application that needs to extract structured information about ongoing events from unstructured texts can take advantage of an OED system.
  arXiv  Detail & Related papers  (2020-07-02T20:44:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.