StreamProfileBench: A Benchmark for Fine-Grained User Profile Inference in Real-World Streaming Scenarios

• URL: http://arxiv.org/abs/2605.25758v2
• Date: Tue, 26 May 2026 03:38:47 GMT
• Title: StreamProfileBench: A Benchmark for Fine-Grained User Profile Inference in Real-World Streaming Scenarios
• Abstract summary: We introduce StreamProfileBench, a benchmark for fine-grained streaming user profiling.<n>We curate a highly authentic dataset of over 120,000 posts from 7,000+ real users across five diverse platforms.<n>By leveraging the temporal correlation of user interests, we propose a novel, annotation-free evaluation framework.
• Score: 38.37032285408029
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large Language Models (LLMs) have reshaped user profiling, yet current evaluations mainly focus on static data snapshots. This paradigm overlooks the reality of personalized systems, where User-Generated Content (UGC) arrives continuously and fine-grained profile evolve rapidly. To bridge this gap, we introduce StreamProfileBench, a large-scale benchmark for fine-grained streaming user profiling. We formalize streaming user profiling as a continuous state maintenance task and curate a highly authentic dataset comprising over 120,000 UGC posts from 7,000+ real users across five diverse platforms. By leveraging the temporal correlation of user interests, we further propose a novel, annotation-free evaluation framework. Extensive experiments across 14 leading LLMs reveal that continuous profile updating remains an open challenge. Models exhibit a systemic conservative bias, over-retaining past interests while failing to recognize interest decay. Ablation experiments further validate the practical utility and necessity of the streaming paradigm.

Related papers

• Less Is Personal: Learning Minimal Sufficient User Profiles for Personalized Language Models [77.02066500576528]
  Enough is a method that iteratively appends behavioral records or emits STOP to construct profiles with adaptive lengths.<n>Enough consistently outperforms strong and retrieval-augmented baselines in both effectiveness and efficiency.
  arXiv  Detail & Related papers  (2026-09-08T03:09:19Z)
• RECAP: Feedback-Driven Streaming Semantic User Profiles for Short-Video Recommendation [74.86562505934961]
  Language-based user profiles convert long behavioral histories into explicit semantic representations for recommendation.<n>We study this problem in real-world short-video recommendation, where user behaviors continuously arrive as streams and profiles must be incrementally updated under limited capacity.<n>We propose RECAP, an offline closed-loop framework for optimizing streaming structured semantic profiles with historical recommendation feedback.
  arXiv  Detail & Related papers  (2026-07-17T08:10:37Z)
• Action-Aware Generative Sequence Modeling for Short Video Recommendation [50.74467504063892]
  Short videos often contain diverse segments, and users may not hold the same attitude toward all of them.<n>Traditional binary-classification recommendation models, which treat a video as a single holistic entity, face limitations in accurately capturing such nuanced preferences.<n>We propose a novel modeling paradigm: Action-Aware Generative Sequence Network (A2Gen), which refines user actions along the temporal dimension.
  arXiv  Detail & Related papers  (2026-04-28T16:41:04Z)
• It's TIME: Towards the Next Generation of Time Series Forecasting Benchmarks [87.7937890373758]
  Time series foundation models (TSFMs) are revolutionizing the forecasting landscape from specific dataset modeling to generalizable task evaluation.<n>We introduce TIME, a next-generation task-centric benchmark comprising 50 fresh datasets and 98 forecasting tasks.<n>We propose a novel pattern-level evaluation perspective that moves beyond traditional dataset-level evaluations based on static meta labels.
  arXiv  Detail & Related papers  (2026-02-12T16:31:01Z)
• Conf-Profile: A Confidence-Driven Reasoning Paradigm for Label-Free User Profiling [20.460209066277777]
  We propose a Confidence-driven Profile reasoning framework Conf-Profile for user profiling.<n>We show that Conf-Profile delivers substantial performance through the two-stage training, improving F1 by 13.97 on Qwen3-8B.
  arXiv  Detail & Related papers  (2025-09-23T09:58:37Z)
• Biases in LLM-Generated Musical Taste Profiles for Recommendation [6.482557558168364]
  Large Language Models (LLMs) for recommendation can generate Natural Language (NL) user taste profiles from consumption data.<n>But it remains unclear whether users consider these profiles to be an accurate representation of their taste.<n>We study this issue in the context of music streaming, where personalization is challenged by a large and culturally diverse catalog.
  arXiv  Detail & Related papers  (2025-07-22T15:44:10Z)
• Towards Explainable Temporal User Profiling with LLMs [3.719862246745416]
  We leverage large language models (LLMs) to generate natural language summaries of users' interaction histories.<n>Our framework not only models temporal user preferences but also produces natural language profiles that can be used to explain recommendations in an interpretable manner.
  arXiv  Detail & Related papers  (2025-05-01T22:02:46Z)
• Know Me, Respond to Me: Benchmarking LLMs for Dynamic User Profiling and Personalized Responses at Scale [53.059480071818136]
  Large Language Models (LLMs) have emerged as personalized assistants for users across a wide range of tasks.<n> PERSONAMEM features curated user profiles with over 180 simulated user-LLM interaction histories.<n>We evaluate LLM chatbots' ability to identify the most suitable response according to the current state of the user's profile.
  arXiv  Detail & Related papers  (2025-04-19T08:16:10Z)
• PersonaX: A Recommendation Agent Oriented User Modeling Framework for Long Behavior Sequence [8.557534799079956]
  PersonaX is an agent-agnostic LLM-UM framework to address these challenges.<n>It augments downstream recommendation agents to achieve better recommendation performance and inference efficiency.
  arXiv  Detail & Related papers  (2025-03-04T08:41:40Z)
• Generative Regression Based Watch Time Prediction for Short-Video Recommendation [36.95095097454143]
  Watch time prediction has emerged as a pivotal task in short video recommendation systems.<n>Recent studies have attempted to address these issues by converting the continuous watch time estimation into an ordinal regression task.<n>We propose a novel Generative Regression (GR) framework that reformulates WTP as a sequence generation task.
  arXiv  Detail & Related papers  (2024-12-28T16:48:55Z)
• Prompt Tuning as User Inherent Profile Inference Machine [68.16976932088708]
  We propose UserIP-Tuning, which uses prompt-tuning to infer user profiles.<n>UserIP-Tuning outperforms state-of-the-art recommendation algorithms.<n>The presented solution has been deployed in Huawei AppGallery's Explore page since May 2025.
  arXiv  Detail & Related papers  (2024-08-13T02:25:46Z)
• Lusifer: LLM-based User SImulated Feedback Environment for online Recommender systems [0.0]
  Reinforcement learning (RL) recommender systems often rely on static datasets that fail to capture the fluid, ever changing nature of user preferences in real-world scenarios.<n>We introduce Lusifer, an LLM-based simulation environment designed to generate dynamic, realistic user feedback for RL-based recommender training.
  arXiv  Detail & Related papers  (2024-05-22T05:43:15Z)
• Contrastive Continual Multi-view Clustering with Filtered Structural Fusion [57.193645780552565]
  Multi-view clustering thrives in applications where views are collected in advance. It overlooks scenarios where data views are collected sequentially, i.e., real-time data. Some methods are proposed to handle it but are trapped in a stability-plasticity dilemma. We propose Contrastive Continual Multi-view Clustering with Filtered Structural Fusion.
  arXiv  Detail & Related papers  (2023-09-26T14:18:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.