From Interaction to Intent: Inferring User Objectives from Provenance Logs

• URL: http://arxiv.org/abs/2607.04501v1
• Date: Sun, 05 Jul 2026 21:02:23 GMT
• Title: From Interaction to Intent: Inferring User Objectives from Provenance Logs
• Abstract summary: We show that provenance logs can be used to classify user intentions in visual exploration tasks.<n>We find that distinct behavioral signatures emerge across different analytic objectives.<n>These findings demonstrate that low-level interaction data can serve as a practical bridge to high-level analytic intent.
• Score: 16.702841059421377
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: The ability to automatically infer analytic intent from user interaction histories could enable interactive AI systems to proactively assist users during exploratory data analysis. In this paper, we examine whether provenance logs -- detailed records capturing sequences and timing of user interactions -- can be used to classify user intentions in visual exploration tasks. To investigate this, we record how participants interact with multiple multidimensional data projections across a range of analytic tasks, capturing fine-grained mouse interaction data throughout each session. We find that distinct behavioral signatures emerge across different analytic objectives. For instance, users examining properties of specific clusters exhibit markedly different interaction patterns compared to those searching for outliers. More importantly, we show that embedding contextual information into interaction provenance enables classifiers to predict user objectives that generalize across datasets and projection methods. These findings demonstrate that low-level interaction data can serve as a practical bridge to high-level analytic intent, contributing to the development of intent-aware visualization systems.

Related papers

• Learning Human-Object Interaction as Groups [52.28258599873394]
  GroupHOI is a framework that propagates contextual information in terms of geometric proximity and semantic similarity.<n>It exhibits leading performance on the more challenging Nonverbal Interaction Detection task.
  arXiv  Detail & Related papers  (2025-10-21T07:25:10Z)
• InterChat: Enhancing Generative Visual Analytics using Multimodal Interactions [22.007942964950217]
  We develop InterChat, a generative visual analytics system that combines direct manipulation of visual elements with natural language inputs.<n>This integration enables precise intent communication and supports progressive, visually driven exploratory data analyses.
  arXiv  Detail & Related papers  (2025-03-06T05:35:19Z)
• Visual-Geometric Collaborative Guidance for Affordance Learning [63.038406948791454]
  We propose a visual-geometric collaborative guided affordance learning network that incorporates visual and geometric cues. Our method outperforms the representative models regarding objective metrics and visual quality.
  arXiv  Detail & Related papers  (2024-10-15T07:35:51Z)
• Spatio-Temporal Context Prompting for Zero-Shot Action Detection [13.22912547389941]
  We propose a method which can effectively leverage the rich knowledge of visual-language models to perform Person-Context Interaction.<n>To address the challenge of recognizing distinct actions by multiple people at the same timestamp, we design the Interest Token Spotting mechanism.<n>Our method achieves superior results compared to previous approaches and can be further extended to multi-action videos.
  arXiv  Detail & Related papers  (2024-08-28T17:59:05Z)
• Enhancing HOI Detection with Contextual Cues from Large Vision-Language Models [56.257840490146]
  ConCue is a novel approach for improving visual feature extraction in HOI detection. We develop a transformer-based feature extraction module with a multi-tower architecture that integrates contextual cues into both instance and interaction detectors.
  arXiv  Detail & Related papers  (2023-11-26T09:11:32Z)
• What User Behaviors Make the Differences During the Process of Visual Analytics? [1.5285292154680246]
  This work presents a study on a comprehensive data collection of user behaviors, and our analysis approach with time-series classification methods. Our user study collects user behaviors on a diverse set of visualization tasks with two comparable systems, desktop and immersive visualizations. Our results reveal that user behaviors can be distinguished during the process of visual analytics and there is a potentially strong association between the physical behaviors of users and the visualization tasks they perform.
  arXiv  Detail & Related papers  (2023-11-01T17:45:52Z)
• Resolving Uncertain Case Identifiers in Interaction Logs: A User Study [0.4014524824655105]
  We propose a neural network-based technique to determine a case notion for click data. We validate its efficacy through a user study based on the segmented event log resulting from interaction data of a mobility sharing company.
  arXiv  Detail & Related papers  (2022-11-21T16:13:04Z)
• A Unified Comparison of User Modeling Techniques for Predicting Data Interaction and Detecting Exploration Bias [17.518601254380275]
  We compare and rank eight user modeling algorithms based on their performance on a diverse set of four user study datasets. Based on our findings, we highlight open challenges and new directions for analyzing user interactions and visualization provenance.
  arXiv  Detail & Related papers  (2022-08-09T19:51:10Z)
• Co-Located Human-Human Interaction Analysis using Nonverbal Cues: A Survey [71.43956423427397]
  We aim to identify the nonverbal cues and computational methodologies resulting in effective performance. This survey differs from its counterparts by involving the widest spectrum of social phenomena and interaction settings. Some major observations are: the most often used nonverbal cue, computational method, interaction environment, and sensing approach are speaking activity, support vector machines, and meetings composed of 3-4 persons equipped with microphones and cameras, respectively.
  arXiv  Detail & Related papers  (2022-07-20T13:37:57Z)
• ConsNet: Learning Consistency Graph for Zero-Shot Human-Object Interaction Detection [101.56529337489417]
  We consider the problem of Human-Object Interaction (HOI) Detection, which aims to locate and recognize HOI instances in the form of human, action, object> in images. We argue that multi-level consistencies among objects, actions and interactions are strong cues for generating semantic representations of rare or previously unseen HOIs. Our model takes visual features of candidate human-object pairs and word embeddings of HOI labels as inputs, maps them into visual-semantic joint embedding space and obtains detection results by measuring their similarities.
  arXiv  Detail & Related papers  (2020-08-14T09:11:18Z)
• Disentangled Graph Collaborative Filtering [100.26835145396782]
  Disentangled Graph Collaborative Filtering (DGCF) is a new model for learning informative representations of users and items from interaction data. By modeling a distribution over intents for each user-item interaction, we iteratively refine the intent-aware interaction graphs and representations. DGCF achieves significant improvements over several state-of-the-art models like NGCF, DisenGCN, and MacridVAE.
  arXiv  Detail & Related papers  (2020-07-03T15:37:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.