Unobtrusive In-Situ Measurement of Behavior Change by Deep Metric Similarity Learning of Motion Patterns

• URL: http://arxiv.org/abs/2509.04174v1
• Date: Thu, 04 Sep 2025 12:46:18 GMT
• Title: Unobtrusive In-Situ Measurement of Behavior Change by Deep Metric Similarity Learning of Motion Patterns
• Authors: Christian Merz, Lukas Schach, Marie Luisa Fiedler, Jean-Luc Lugrin, Carolin Wienrich, Marc Erich Latoschik,
• Abstract summary: This paper introduces an in-situ measurement method to detect user behavior changes during arbitrary exposures in XR systems.<n>We present a biometric user model based on deep metric similarity learning, which uses high-dimensional embeddings as reference vectors.
• Score: 25.896192890947216
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: This paper introduces an unobtrusive in-situ measurement method to detect user behavior changes during arbitrary exposures in XR systems. Here, such behavior changes are typically associated with the Proteus effect or bodily affordances elicited by different avatars that the users embody in XR. We present a biometric user model based on deep metric similarity learning, which uses high-dimensional embeddings as reference vectors to identify behavior changes of individual users. We evaluate our model against two alternative approaches: a (non-learned) motion analysis based on central tendencies of movement patterns and subjective post-exposure embodiment questionnaires frequently used in various XR exposures. In a within-subject study, participants performed a fruit collection task while embodying avatars of different body heights (short, actual-height, and tall). Subjective assessments confirmed the effective manipulation of perceived body schema, while the (non-learned) objective analyses of head and hand movements revealed significant differences across conditions. Our similarity learning model trained on the motion data successfully identified the elicited behavior change for various query and reference data pairings of the avatar conditions. The approach has several advantages in comparison to existing methods: 1) In-situ measurement without additional user input, 2) generalizable and scalable motion analysis for various use cases, 3) user-specific analysis on the individual level, and 4) with a trained model, users can be added and evaluated in real time to study how avatar changes affect behavior.

Related papers

• PersonaDrift: A Benchmark for Temporal Anomaly Detection in Language-Based Dementia Monitoring [0.9668407688201359]
  PersonaDrift is a benchmark designed to evaluate machine learning and statistical methods for detecting progressive changes in daily communication.<n>The benchmark focuses on two forms of longitudinal change that caregivers highlighted as particularly salient: flattened sentiment and off-topic replies.<n>Preliminary results show that flattened sentiment can often be detected with simple statistical models in users with low baseline variability.
  arXiv  Detail & Related papers  (2025-11-20T15:15:00Z)
• Evaluating Multiview Object Consistency in Humans and Image Models [68.36073530804296]
  We leverage an experimental design from the cognitive sciences which requires zero-shot visual inferences about object shape. We collect 35K trials of behavioral data from over 500 participants. We then evaluate the performance of common vision models.
  arXiv  Detail & Related papers  (2024-09-09T17:59:13Z)
• Disentangled Interaction Representation for One-Stage Human-Object Interaction Detection [70.96299509159981]
  Human-Object Interaction (HOI) detection is a core task for human-centric image understanding. Recent one-stage methods adopt a transformer decoder to collect image-wide cues that are useful for interaction prediction. Traditional two-stage methods benefit significantly from their ability to compose interaction features in a disentangled and explainable manner.
  arXiv  Detail & Related papers  (2023-12-04T08:02:59Z)
• What User Behaviors Make the Differences During the Process of Visual Analytics? [1.5285292154680246]
  This work presents a study on a comprehensive data collection of user behaviors, and our analysis approach with time-series classification methods. Our user study collects user behaviors on a diverse set of visualization tasks with two comparable systems, desktop and immersive visualizations. Our results reveal that user behaviors can be distinguished during the process of visual analytics and there is a potentially strong association between the physical behaviors of users and the visualization tasks they perform.
  arXiv  Detail & Related papers  (2023-11-01T17:45:52Z)
• DCID: Deep Canonical Information Decomposition [84.59396326810085]
  We consider the problem of identifying the signal shared between two one-dimensional target variables. We propose ICM, an evaluation metric which can be used in the presence of ground-truth labels. We also propose Deep Canonical Information Decomposition (DCID) - a simple, yet effective approach for learning the shared variables.
  arXiv  Detail & Related papers  (2023-06-27T16:59:06Z)
• Causal Estimation of User Learning in Personalized Systems [5.016998307223021]
  We introduce a non-parametric causal model of user actions in a personalized system. We show that the Cookie-Cookie-Day experiment, designed for the measurement of the user learning effect, is biased when there is personalization. We derive new experimental designs that intervene in the personalization system to generate the variation necessary to separately identify the causal effect mediated through user learning and personalization.
  arXiv  Detail & Related papers  (2023-06-01T09:37:43Z)
• Self-similarity Driven Scale-invariant Learning for Weakly Supervised Person Search [66.95134080902717]
  We propose a novel one-step framework, named Self-similarity driven Scale-invariant Learning (SSL) We introduce a Multi-scale Exemplar Branch to guide the network in concentrating on the foreground and learning scale-invariant features. Experiments on PRW and CUHK-SYSU databases demonstrate the effectiveness of our method.
  arXiv  Detail & Related papers  (2023-02-25T04:48:11Z)
• Impact of Feedback Type on Explanatory Interactive Learning [4.039245878626345]
  Explanatory Interactive Learning (XIL) collects user feedback on visual model explanations to implement a Human-in-the-Loop (HITL) based interactive learning scenario. We compare the effectiveness of two different user feedback types in image classification tasks. We show that identifying and annotating spurious image features that a model finds salient results in superior classification and explanation accuracy than user feedback that tells a model to focus on valid image features.
  arXiv  Detail & Related papers  (2022-09-26T07:33:54Z)
• View-Invariant Skeleton-based Action Recognition via Global-Local Contrastive Learning [15.271862140292837]
  We propose a new view-invariant representation learning approach, without any manual action labeling, for skeleton-based human action recognition. We leverage the multi-view skeleton data simultaneously taken for the same person in the network training, by maximizing the mutual information between the representations extracted from different views. We show that the proposed method is robust to the view difference of the input skeleton data and significantly boosts the performance of unsupervised skeleton-based human action methods.
  arXiv  Detail & Related papers  (2022-09-23T15:00:57Z)
• Interpretable Multi-dataset Evaluation for Named Entity Recognition [110.64368106131062]
  We present a general methodology for interpretable evaluation for the named entity recognition (NER) task. The proposed evaluation method enables us to interpret the differences in models and datasets, as well as the interplay between them. By making our analysis tool available, we make it easy for future researchers to run similar analyses and drive progress in this area.
  arXiv  Detail & Related papers  (2020-11-13T10:53:27Z)
• End-to-End Models for the Analysis of System 1 and System 2 Interactions based on Eye-Tracking Data [99.00520068425759]
  We propose a computational method, within a modified visual version of the well-known Stroop test, for the identification of different tasks and potential conflicts events. A statistical analysis shows that the selected variables can characterize the variation of attentive load within different scenarios. We show that Machine Learning techniques allow to distinguish between different tasks with a good classification accuracy.
  arXiv  Detail & Related papers  (2020-02-03T17:46:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.