HorizonBench: Long-Horizon Personalization with Evolving Preferences

• URL: http://arxiv.org/abs/2604.17283v1
• Date: Sun, 19 Apr 2026 06:55:10 GMT
• Title: HorizonBench: Long-Horizon Personalization with Evolving Preferences
• Abstract summary: We introduce a data generator that produces conversations from a structured mental state graph.<n>From it construct HorizonBench, a benchmark of 4,245 items from 360 simulated users with 6-month conversation histories.<n> HorizonBench provides a testbed for long-context modeling, memory-augmented architectures, theory-of-mind reasoning, and user modeling.
• Score: 73.32940850193681
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: User preferences evolve across months of interaction, and tracking them requires inferring when a stated preference has been changed by a subsequent life event. We define this problem as long-horizon personalization and observe that progress on it is limited by data availability and measurement, with no existing resource providing both naturalistic long-horizon interactions and the ground-truth provenance needed to diagnose why models fail. We introduce a data generator that produces conversations from a structured mental state graph, yielding ground-truth provenance for every preference change across 6-month timelines, and from it construct HorizonBench, a benchmark of 4,245 items from 360 simulated users with 6-month conversation histories averaging ~4,300 turns and ~163K tokens. HorizonBench provides a testbed for long-context modeling, memory-augmented architectures, theory-of-mind reasoning, and user modeling. Across 25 frontier models, the best model reaches 52.8% and most score at or below the 20% chance baseline. When these models err on evolved preferences, over a third of the time they select the user's originally stated value without tracking the updated user state. This belief-update failure persists across context lengths and expression explicitness levels, identifying state-tracking capability as the primary bottleneck for long-horizon personalization.

Related papers

• PHASE-Tree: Modeling Character-State Evolution in Long-Horizon Role-Playing Dialogue [37.64180837016861]
  PHASE-Tree is a multi-timescale character-state tree with an immutable identity root and mutable persona, session, and moment layers.<n>It conditions generation through explicit textual provision or implicit parametric adaptation.<n>On the long-dialogue core, PHASE-Tree ranks first in 11 of 12 dataset-metric cells against internal variants and all 12 cells against external textual baselines.
  arXiv  Detail & Related papers  (2026-08-07T08:50:14Z)
• Rushes: A Human Preference Dataset for Pluralistic Alignment [11.198313470367365]
  Rushes is a dataset and benchmark for studying revealed human engagement preferences in interactive narrative environments.<n>It contains 44,226 decision events from 8,167 unique users across six games.<n>We show that user choices exhibit structured, non-random patterns, quantified by a low choice entropy relative to a uniform baseline.
  arXiv  Detail & Related papers  (2026-07-22T22:32:34Z)
• BehaviorBench: Modeling Real-World User Decisions from Behavioral Traces [53.55407626944686]
  textscBehaviorBench is a benchmark for evaluating personalized decision modeling from real-world behavioral traces.<n>textscBehaviorBench reconstructs wallet-level decision histories from observed public prediction-market and on-chain records.<n>Across 2,000 evaluation wallets, the benchmark contains 141,445 Belief instances and 1,485,972 Trade instances.
  arXiv  Detail & Related papers  (2026-06-01T19:04:36Z)
• LongDS-Bench: On the Failure of Long-Horizon Agentic Data Analysis [29.918718173329037]
  LongDS is a benchmark for long-horizon, multi-turn data analysis.<n>Agents must maintain, update, restore, and compose evolving analytical states.<n>LongDS comprises 68 tasks constructed from real-world Kaggle notebooks.
  arXiv  Detail & Related papers  (2026-05-28T18:00:20Z)
• Response Time Enhances Alignment with Heterogeneous Preferences [49.69696266152175]
  We show that augmenting preference datasets with a simple, secondary signal can restore identifiability of the population's average preference.<n>Our results bring promises and open up new opportunities for future data-collection pipelines.
  arXiv  Detail & Related papers  (2026-05-07T22:05:23Z)
• HORIZON: A Benchmark for In-the-wild User Behaviour Modeling [14.815729073163729]
  We present HORIZON, a new benchmark that reformulates user modeling along three axes i.e. dataset, task, and evaluation.<n>Built from a large-scale, cross-domain reformulation of Amazon Reviews, HORIZON covers 54M users and 35M items.<n>Unlike prior benchmarks, it challenges models to generalize across domains, users, and time, moving beyond standard missing-positive prediction in the same domain.
  arXiv  Detail & Related papers  (2026-04-19T04:45:48Z)
• A Grid-Based Framework for E-Scooter Demand Representation and Temporal Input Design for Deep Learning: Evidence from Austin, Texas [0.17188280334580194]
  This paper introduces a statistically grounded method for designing temporal input structures for image-to-image demand prediction.<n>We build a grid-based census dataset by converting trip records into hourly pickup and dropoff demand images.<n>The proposed design reduces mean squared error by up to 37 percent for next-hour prediction and 35 percent for next-24-hour prediction.
  arXiv  Detail & Related papers  (2026-03-13T21:36:04Z)
• It's TIME: Towards the Next Generation of Time Series Forecasting Benchmarks [87.7937890373758]
  Time series foundation models (TSFMs) are revolutionizing the forecasting landscape from specific dataset modeling to generalizable task evaluation.<n>We introduce TIME, a next-generation task-centric benchmark comprising 50 fresh datasets and 98 forecasting tasks.<n>We propose a novel pattern-level evaluation perspective that moves beyond traditional dataset-level evaluations based on static meta labels.
  arXiv  Detail & Related papers  (2026-02-12T16:31:01Z)
• ALPBench: A Benchmark for Attribution-level Long-term Personal Behavior Understanding [53.88804678012327]
  ALPBench is a Benchmark for Attribution-level Long-term Personal Behavior Understanding.<n>It predicts user-interested attribute combinations, enabling ground-truth evaluation.<n>It models preferences from long-term historical behaviors rather than users' explicitly expressed requests.
  arXiv  Detail & Related papers  (2026-02-03T03:32:16Z)
• SPRInG: Continual LLM Personalization via Selective Parametric Adaptation and Retrieval-Interpolated Generation [11.253466187411524]
  We introduce SPRInG, a novel semi-parametric framework designed for effective continual personalization.<n>During training, SPRInG employs drift-driven selective adaptation, which utilizes a likelihood-based scoring function to identify high-novelty interactions.<n>Experiments on the long-form personalized generation benchmark demonstrate that SPRInG outperforms existing baselines.
  arXiv  Detail & Related papers  (2026-01-15T01:32:27Z)
• GRAID: Enhancing Spatial Reasoning of VLMs Through High-Fidelity Data Generation [31.365285503503475]
  We present a framework for learning spatial reasoning using 2D boxes from standard detectors.<n>We show that when trained on GRAID data, models learn spatial reasoning concepts that generalize on over-detailed held-out types.<n>We also show that when trained on all questions types, achieve improvements on several existing benchmarks.
  arXiv  Detail & Related papers  (2025-10-25T02:07:23Z)
• Discrete-event Tensor Factorization: Learning a Smooth Embedding for Continuous Domains [0.0]
  This paper analyzes how time can be encoded in factorization-style recommendation models.<n>By including absolute time as a feature, our models can learn varying user preferences and changing item perception over time.
  arXiv  Detail & Related papers  (2025-08-06T08:54:57Z)
• OmniTraj: Pre-Training on Heterogeneous Data for Adaptive and Zero-Shot Human Trajectory Prediction [62.385417528148224]
  We present OmniTraj, a Transformer-based model pre-trained on a large-scale, heterogeneous dataset.<n>Experiments show that explicitly conditioning on the frame rate enables OmniTraj to achieve state-of-the-art zero-shot transfer performance.
  arXiv  Detail & Related papers  (2025-07-31T15:37:09Z)
• Output Scaling: YingLong-Delayed Chain of Thought in a Large Pretrained Time Series Forecasting Model [55.25659103706409]
  This framework achieves state-of-the-art performance for our designed foundation model, YingLong.<n>YingLong is a non-causal, bidirectional attention encoder-only transformer trained through masked token recovery.<n>We release four foundation models ranging from 6M to 300M parameters, demonstrating superior results in zero-shot tasks.
  arXiv  Detail & Related papers  (2025-05-20T14:31:06Z)
• Learning Transferrable Parameters for Long-tailed Sequential User Behavior Modeling [70.64257515361972]
  We argue that focusing on tail users could bring more benefits and address the long tails issue. Specifically, we propose a gradient alignment and adopt an adversarial training scheme to facilitate knowledge transfer from the head to the tail.
  arXiv  Detail & Related papers  (2020-10-22T03:12:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.