Modeling Behaviour to Predict User State: Self-Reports as Ground Truth

• URL: http://arxiv.org/abs/2007.14461v1
• Date: Tue, 28 Jul 2020 20:09:09 GMT
• Title: Modeling Behaviour to Predict User State: Self-Reports as Ground Truth
• Abstract summary: Methods that detect user states such as emotions are useful for interactive systems. We argue for model-based approaches that are trained on user behaviour and self-reported user state as ground truths.
• Score: 28.327411414505306
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Methods that detect user states such as emotions are useful for interactive systems. In this position paper, we argue for model-based approaches that are trained on user behaviour and self-reported user state as ground truths. In an application context, they record behaviour, extract relevant features, and use the models to predict user states. We describe how this approach can be implemented and discuss its benefits in comparison to solely self-reports in an application and to models of behaviour without the selfreport ground truths. Finally, we discuss shortcomings of this approach by considering its drawbacks and limitations.

Related papers

• Learning Dynamic User Personas from Implicit Interaction Streams via Iterative Refinement [4.027292457517535]
  We propose IRIS, a framework that learns dynamic user personas directly from implicit interaction streams.<n>IRIS produces stable personas and distinguishes individual users while revealing limitations of memory-only approaches.<n>We validate IRIS on anonymized real-world Reddit r/AmItheAsshole (AITA) data, with personas built solely from each author's historical interactions.
  arXiv  Detail & Related papers  (2026-07-29T04:56:14Z)
• BehaviorBench: Modeling Real-World User Decisions from Behavioral Traces [53.55407626944686]
  textscBehaviorBench is a benchmark for evaluating personalized decision modeling from real-world behavioral traces.<n>textscBehaviorBench reconstructs wallet-level decision histories from observed public prediction-market and on-chain records.<n>Across 2,000 evaluation wallets, the benchmark contains 141,445 Belief instances and 1,485,972 Trade instances.
  arXiv  Detail & Related papers  (2026-06-01T19:04:36Z)
• Know You Before You Speak: User-State Modeling for LLM Personalization in Multi-Turn Conversation [40.40960192827234]
  We propose PUMA, a framework that formulates personalization as decision-making under partial observability.<n>Puma maintains a belief over the user's hidden state, refines the user state model for observation generation and action-conditioned state transition.<n>Experiments show that PUMA improves long-horizon dialogue outcomes while maintaining strong response quality.
  arXiv  Detail & Related papers  (2026-05-23T16:28:40Z)
• From Feelings to Metrics: Understanding and Formalizing How Users Vibe-Test LLMs [48.36707944399574]
  We study how vibe-testing works in practice and then formalize it to support systematic analysis.<n>In experiments on coding benchmarks, we find that combining personalized prompts and user-aware evaluation can change which model is preferred.
  arXiv  Detail & Related papers  (2026-04-15T17:57:08Z)
• Aligning Language Models from User Interactions [41.288783114966996]
  We propose a principled and scalable method for learning directly from user interactions through self-distillation.<n>We show that training on real-world user conversations from WildChat improves language models across standard alignment and instruction-following benchmarks.
  arXiv  Detail & Related papers  (2026-02-18T16:31:35Z)
• The Inadequacy of Offline LLM Evaluations: A Need to Account for Personalization in Model Behavior [32.02851847409678]
  We show that identical benchmark questions to the same language model can produce markedly different responses when prompted to a state-less system.<n>We compare offline evaluations to field evaluations conducted by having 800 real users of ChatGPT and Gemini pose benchmark and other provided questions to their chat interfaces.
  arXiv  Detail & Related papers  (2025-09-18T20:41:20Z)
• EdgeWisePersona: A Dataset for On-Device User Profiling from Natural Language Interactions [0.6650227510403052]
  This paper introduces a novel dataset designed to assess and improve small language models deployable on edge devices.<n>At the core of the dataset are structured user profiles, each defined by a set of routines.<n>A large language model (LLM) generates corresponding interaction sessions that simulate realistic, diverse, and context-aware dialogues.
  arXiv  Detail & Related papers  (2025-05-16T16:29:21Z)
• Where is this coming from? Making groundedness count in the evaluation of Document VQA models [12.951716701565019]
  We argue that common evaluation metrics do not account for the semantic and multimodal groundedness of a model's outputs. We propose a new evaluation methodology that accounts for the groundedness of predictions. Our proposed methodology is parameterized in such a way that users can configure the score according to their preferences.
  arXiv  Detail & Related papers  (2025-03-24T20:14:46Z)
• USE: Dynamic User Modeling with Stateful Sequence Models [26.74966828348815]
  User Stateful Embedding (USE) generates user embeddings without the need for exhaustive reprocessing. We introduce a novel training objective named future W-behavior prediction to transcend the limitations of next-token prediction. We conduct experiments on 8 downstream tasks using Snapchat users' behavioral logs in both static (i.e., fixed user behavior sequences) and dynamic (i.e. periodically updated user behavior sequences) settings.
  arXiv  Detail & Related papers  (2024-03-20T07:05:19Z)
• TETRIS: Towards Exploring the Robustness of Interactive Segmentation [39.1981941213761]
  We propose a methodology for finding extreme user inputs by a direct optimization in a white-box adversarial attack on the interactive segmentation model. We report the results of an extensive evaluation of dozens of models.
  arXiv  Detail & Related papers  (2024-02-09T01:36:21Z)
• What if you said that differently?: How Explanation Formats Affect Human Feedback Efficacy and User Perception [53.4840989321394]
  We analyze the effect of rationales generated by QA models to support their answers. We present users with incorrect answers and corresponding rationales in various formats. We measure the effectiveness of this feedback in patching these rationales through in-context learning.
  arXiv  Detail & Related papers  (2023-11-16T04:26:32Z)
• Bring Your Own Data! Self-Supervised Evaluation for Large Language Models [52.15056231665816]
  We propose a framework for self-supervised evaluation of Large Language Models (LLMs) We demonstrate self-supervised evaluation strategies for measuring closed-book knowledge, toxicity, and long-range context dependence. We find strong correlations between self-supervised and human-supervised evaluations.
  arXiv  Detail & Related papers  (2023-06-23T17:59:09Z)
• Federated Privacy-preserving Collaborative Filtering for On-Device Next App Prediction [52.16923290335873]
  We propose a novel SeqMF model to solve the problem of predicting the next app launch during mobile device usage. We modify the structure of the classical matrix factorization model and update the training procedure to sequential learning. One more ingredient of the proposed approach is a new privacy mechanism that guarantees the protection of the sent data from the users to the remote server.
  arXiv  Detail & Related papers  (2023-02-05T10:29:57Z)
• Latent User Intent Modeling for Sequential Recommenders [92.66888409973495]
  Sequential recommender models learn to predict the next items a user is likely to interact with based on his/her interaction history on the platform. Most sequential recommenders however lack a higher-level understanding of user intents, which often drive user behaviors online. Intent modeling is thus critical for understanding users and optimizing long-term user experience.
  arXiv  Detail & Related papers  (2022-11-17T19:00:24Z)
• Canary in a Coalmine: Better Membership Inference with Ensembled Adversarial Queries [53.222218035435006]
  We use adversarial tools to optimize for queries that are discriminative and diverse. Our improvements achieve significantly more accurate membership inference than existing methods.
  arXiv  Detail & Related papers  (2022-10-19T17:46:50Z)
• You Mostly Walk Alone: Analyzing Feature Attribution in Trajectory Prediction [52.442129609979794]
  Recent deep learning approaches for trajectory prediction show promising performance. It remains unclear which features such black-box models actually learn to use for making predictions. This paper proposes a procedure that quantifies the contributions of different cues to model performance.
  arXiv  Detail & Related papers  (2021-10-11T14:24:15Z)
• Intellige: A User-Facing Model Explainer for Narrative Explanations [0.0]
  We propose Intellige, a user-facing model explainer that creates user-digestible interpretations and insights. Intellige builds an end-to-end pipeline from machine learning platforms to end user platforms.
  arXiv  Detail & Related papers  (2021-05-27T05:11:47Z)
• On Variational Inference for User Modeling in Attribute-Driven Collaborative Filtering [10.64460581091531]
  We present an approach to use causal inference to learn user-attribute affinities through temporal contexts. We formulate this objective as a Probabilistic Machine Learning problem and apply a variational inference based method to estimate the model parameters.
  arXiv  Detail & Related papers  (2020-12-02T22:39:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.