Talking to Your Data: Exploring Embodied Conversation as an Interface for Personal Health Reflection

• URL: http://arxiv.org/abs/2606.17767v1
• Date: Tue, 16 Jun 2026 10:34:27 GMT
• Title: Talking to Your Data: Exploring Embodied Conversation as an Interface for Personal Health Reflection
• Abstract summary: We present a system that combines lightweight preprocessing of wearable data with a Unity-based embodied character.<n>We evaluate this approach through a simulated-self user study using a within-subject design.
• Score: 3.887490901679446
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Personal health data from wearables are typically presented through dashboards of charts and summary statistics, requiring users to actively interpret patterns and implications. We explore an alternative interaction paradigm: engaging with personal health data through an embodied conversational agent that facilitates objective data reflection in dialogue with the user. We present a system that combines lightweight preprocessing of wearable data with a Unity-based embodied character. Internally, the system follows a dual-agent design in which an Observer agent extracts descriptive statistics and temporal trends, and a Presenter agent communicates these findings through "spoken statistics," intentionally refraining from clinical advice to isolate the impact of the interaction modality. We evaluate this approach through a simulated-self user study (N=5) using a within-subject design. Participants adopted health personas and goals derived from the LifeSnaps dataset to compare traditional dashboard exploration with embodied conversational reflection. Our evaluation focuses on perceived understanding, the specificity of generated actions, and the cognitive shift from passive viewing to active sensemaking. The paper contributes a functional prototype, a design pattern for objective health data narrative generation, and early empirical insights into how embodiment affects the interpretation of personal health metrics.

Related papers

• Human-Inspired Social Engagement Analysis via Interpretable Mutual Visual Attention [7.72838047070244]
  We propose an interpretable computational model of social engagement inspired by psychological theories of mutual visual attention.<n>Our framework explicitly models dyadic visual attention and aggregates these cues into interpretable measures of individual and group engagement.
  arXiv  Detail & Related papers  (2026-08-25T14:05:35Z)
• Setoka: A Benchmark for Hierarchical User Understanding in Personalized Agents over Heterogeneous Data [8.564332409420473]
  We propose Setoka, a benchmark for evaluating memory-augmented personalized agents.<n>Setoka defines four levels of user understanding, i.e., semantic memory, episodic memory, behavior pattern, and personality trait.<n>Our comprehensive evaluation reveals that while existing systems perform well on semantic memory retrieval, their performance declines on episodic memory.
  arXiv  Detail & Related papers  (2026-07-29T15:47:40Z)
• From Interaction to Intent: Inferring User Objectives from Provenance Logs [16.702841059421377]
  We show that provenance logs can be used to classify user intentions in visual exploration tasks.<n>We find that distinct behavioral signatures emerge across different analytic objectives.<n>These findings demonstrate that low-level interaction data can serve as a practical bridge to high-level analytic intent.
  arXiv  Detail & Related papers  (2026-07-05T21:02:23Z)
• Personal Visual Context Learning in Large Multimodal Models [53.33351452841322]
  We present Personal Visual Context Learning (Personal ), the prompt-time capability of using user-specific visual context to resolve personalized queries.<n>Our analysis of LMMs identifies a profound context utilization gap, revealing that the mechanisms for leveraging visual evidence, as well as aggregating multiple visual observations, remain critically understudied.<n>Motivated by these findings, we propose the Agentic Context Bank, a strong-time baseline that structures a user's visual context into a self-refining memory bank and employs query-adaptive evidence selection.
  arXiv  Detail & Related papers  (2026-05-11T17:59:17Z)
• PersonaDrift: A Benchmark for Temporal Anomaly Detection in Language-Based Dementia Monitoring [0.9668407688201359]
  PersonaDrift is a benchmark designed to evaluate machine learning and statistical methods for detecting progressive changes in daily communication.<n>The benchmark focuses on two forms of longitudinal change that caregivers highlighted as particularly salient: flattened sentiment and off-topic replies.<n>Preliminary results show that flattened sentiment can often be detected with simple statistical models in users with low baseline variability.
  arXiv  Detail & Related papers  (2025-11-20T15:15:00Z)
• Modeling User Preferences via Brain-Computer Interfacing [54.3727087164445]
  We use Brain-Computer Interfacing technology to infer users' preferences, their attentional correlates towards visual content, and their associations with affective experience. We link these to relevant applications, such as information retrieval, personalized steering of generative models, and crowdsourcing population estimates of affective experiences.
  arXiv  Detail & Related papers  (2024-05-15T20:41:46Z)
• Revisiting Self-supervised Learning of Speech Representation from a Mutual Information Perspective [68.20531518525273]
  We take a closer look into existing self-supervised methods of speech from an information-theoretic perspective. We use linear probes to estimate the mutual information between the target information and learned representations. We explore the potential of evaluating representations in a self-supervised fashion, where we estimate the mutual information between different parts of the data without using any labels.
  arXiv  Detail & Related papers  (2024-01-16T21:13:22Z)
• Probabilistic emotion and sentiment modelling of patient-reported experiences [0.04096453902709291]
  This study introduces a novel methodology for modelling patient emotions from online patient experience narratives. We employ metadata network topic modelling to analyse patient-reported experiences from Care Opinion. We develop a probabilistic, context-specific emotion recommender system capable of predicting both multilabel emotions and binary sentiments.
  arXiv  Detail & Related papers  (2024-01-09T05:39:20Z)
• Disentangled Interaction Representation for One-Stage Human-Object Interaction Detection [70.96299509159981]
  Human-Object Interaction (HOI) detection is a core task for human-centric image understanding. Recent one-stage methods adopt a transformer decoder to collect image-wide cues that are useful for interaction prediction. Traditional two-stage methods benefit significantly from their ability to compose interaction features in a disentangled and explainable manner.
  arXiv  Detail & Related papers  (2023-12-04T08:02:59Z)
• Enhancing HOI Detection with Contextual Cues from Large Vision-Language Models [56.257840490146]
  ConCue is a novel approach for improving visual feature extraction in HOI detection. We develop a transformer-based feature extraction module with a multi-tower architecture that integrates contextual cues into both instance and interaction detectors.
  arXiv  Detail & Related papers  (2023-11-26T09:11:32Z)
• MedNgage: A Dataset for Understanding Engagement in Patient-Nurse Conversations [4.847266237348932]
  Patients who effectively manage their symptoms often demonstrate higher levels of engagement in conversations and interventions with healthcare practitioners. It is crucial for AI systems to understand the engagement in natural conversations between patients and practitioners to better contribute toward patient care. We present a novel dataset (MedNgage) which consists of patient-nurse conversations about cancer symptom management.
  arXiv  Detail & Related papers  (2023-05-31T16:06:07Z)
• Co-Located Human-Human Interaction Analysis using Nonverbal Cues: A Survey [71.43956423427397]
  We aim to identify the nonverbal cues and computational methodologies resulting in effective performance. This survey differs from its counterparts by involving the widest spectrum of social phenomena and interaction settings. Some major observations are: the most often used nonverbal cue, computational method, interaction environment, and sensing approach are speaking activity, support vector machines, and meetings composed of 3-4 persons equipped with microphones and cameras, respectively.
  arXiv  Detail & Related papers  (2022-07-20T13:37:57Z)
• Context-Aware Personality Inference in Dyadic Scenarios: Introducing the UDIVA Dataset [21.140721329446595]
  This paper introduces UDIVA, a new non-acted dataset of face-to-face dyadic interactions. The dataset consists of 90.5 hours of socio-dyadic interactions among 147 participants distributed in 188 sessions. It includes self-demographic, self- and peer-reported personality, internal state, and relationship profiling from participants.
  arXiv  Detail & Related papers  (2020-12-28T15:08:02Z)
• You Impress Me: Dialogue Generation via Mutual Persona Perception [62.89449096369027]
  The research in cognitive science suggests that understanding is an essential signal for a high-quality chit-chat conversation. Motivated by this, we propose P2 Bot, a transmitter-receiver based framework with the aim of explicitly modeling understanding.
  arXiv  Detail & Related papers  (2020-04-11T12:51:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.