Adopt $\neq$ Adapt: Longitudinal Analyses of LLM Conversations in the Wild

• URL: http://arxiv.org/abs/2605.29018v1
• Date: Wed, 27 May 2026 19:17:25 GMT
• Title: Adopt $\neq$ Adapt: Longitudinal Analyses of LLM Conversations in the Wild
• Abstract summary: We analyze the conversational trajectories of $sim$12,000 randomly sampled Microsoft Bing Copilot users.<n>We find that trends in individual user trajectories are much weaker.<n>More active users have more successful conversations and use the LLM for more complex and professionally oriented tasks.
• Score: 5.263910852465185
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Although a growing body of research has begun to describe user--LLM interactions, the picture it paints is largely static; little is known about how individual users change their behavior over time. To address this gap, we analyze the conversational trajectories of $\sim$12,000 randomly sampled Microsoft Bing Copilot users and compare these with data from WildChat-4.8M. While the Copilot data contains significant population-level trends, we find that trends in individual user trajectories are much weaker; user habits prove to be overwhelmingly sticky. We also find stark differences between users of different activity levels: more active users have more successful conversations and use the LLM for more complex and professionally oriented tasks. Some user trends also appear in WildChat-4.8M, but we find evidence that this dataset is significantly skewed towards highly proficient "power" users. Ultimately, our results suggest that existing user behavior is difficult to change and demonstrate the extent of user heterogeneity. Our comparison between datasets highlights that WildChat does not represent typical user-AI interactions, an important caveat for downstream uses of the data.

Related papers

• Priming, Path-dependence, and Plasticity: Understanding the molding of user-LLM interaction and its implications from (many) chat logs in the wild [14.896858577447093]
  This work explores a new approach to studying real-world user-LLM interactions through large-scale chat logs from the wild.<n>We demonstrate key patterns in user expressions despite varied tasks.
  arXiv  Detail & Related papers  (2026-05-07T07:00:25Z)
• Learning Next Action Predictors from Human-Computer Interaction [53.41504288073184]
  LongNAP is a user model that combines parametric and in-context learning to reason over long interaction histories.<n>We label over 360K actions across one month of continuous phone usage from 20 users, amounting to 1,800 hours of screen time.<n>LongNAP significantly outperforms supervised finetuning and prompted baselines on held-out data.
  arXiv  Detail & Related papers  (2026-03-06T05:31:04Z)
• Towards Realistic Personalization: Evaluating Long-Horizon Preference Following in Personalized User-LLM Interactions [50.70965714314064]
  Large Language Models (LLMs) are increasingly serving as personal assistants, where users share complex and diverse preferences over extended interactions.<n>This work proposes RealPref, a benchmark for evaluating realistic preference-following in personalized user-LLM interactions.
  arXiv  Detail & Related papers  (2026-03-04T15:42:43Z)
• RoGBot: Relationship-Oblivious Graph-based Neural Network with Contextual Knowledge for Bot Detection [3.884231159866055]
  We propose a novel framework that integrates detailed textual features with enriched user metadata.<n>Our method uses transformer-based models (e.g., BERT) to extract deep semantic embeddings from tweets.<n> Experimental results on the Cresci-15, Cresci-17, and PAN 2019 datasets demonstrate the robustness of our approach.
  arXiv  Detail & Related papers  (2025-10-25T05:14:58Z)
• Addressing Personalized Bias for Unbiased Learning to Rank [56.663619153713434]
  Unbiased learning to rank (ULTR) aims to learn unbiased ranking models from biased user behavior logs.<n>We propose a novel user-aware inverse-propensity-score estimator for learning-to-rank objectives.
  arXiv  Detail & Related papers  (2025-08-28T14:01:31Z)
• User Feedback in Human-LLM Dialogues: A Lens to Understand Users But Noisy as a Learning Signal [59.120335322495436]
  We analyze user feedback in the user-LLM conversation logs, providing insights into when and why such feedback occurs.<n>Second, we study harvesting learning signals from such implicit user feedback.
  arXiv  Detail & Related papers  (2025-07-30T23:33:29Z)
• Prototypical Human-AI Collaboration Behaviors from LLM-Assisted Writing in the Wild [10.23533525266164]
  Large language models (LLMs) are used in complex writing to steer generations to better fit their needs.<n>We conduct a large-scale analysis of this collaborative behavior for users engaged in writing tasks in the wild.<n>We identify prototypical behaviors in how users interact with LLMs in prompts following their original request.
  arXiv  Detail & Related papers  (2025-05-21T21:13:01Z)
• Know Me, Respond to Me: Benchmarking LLMs for Dynamic User Profiling and Personalized Responses at Scale [53.059480071818136]
  Large Language Models (LLMs) have emerged as personalized assistants for users across a wide range of tasks.<n> PERSONAMEM features curated user profiles with over 180 simulated user-LLM interaction histories.<n>We evaluate LLM chatbots' ability to identify the most suitable response according to the current state of the user's profile.
  arXiv  Detail & Related papers  (2025-04-19T08:16:10Z)
• Mind the Gap! Static and Interactive Evaluations of Large Audio Models [55.87220295533817]
  Large Audio Models (LAMs) are designed to power voice-native experiences.<n>This study introduces an interactive approach to evaluate LAMs and collect 7,500 LAM interactions from 484 participants.
  arXiv  Detail & Related papers  (2025-02-21T20:29:02Z)
• WildChat: 1M ChatGPT Interaction Logs in the Wild [88.05964311416717]
  WildChat is a corpus of 1 million user-ChatGPT conversations, which consists of over 2.5 million interaction turns. In addition to timestamped chat transcripts, we enrich the dataset with demographic data, including state, country, and hashed IP addresses.
  arXiv  Detail & Related papers  (2024-05-02T17:00:02Z)
• DIGMN: Dynamic Intent Guided Meta Network for Differentiated User Engagement Forecasting in Online Professional Social Platforms [32.70471436337077]
  A major reason for the differences in user engagement patterns is that users have different intents. We propose a Dynamic Guided Meta Network (DIGMN) which can explicitly model user intent varying with time. Our method outperforms state-of-the-art baselines significantly.
  arXiv  Detail & Related papers  (2022-10-22T09:57:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.