Rushes: A Human Preference Dataset for Pluralistic Alignment

• URL: http://arxiv.org/abs/2607.20767v1
• Date: Wed, 22 Jul 2026 22:32:34 GMT
• Title: Rushes: A Human Preference Dataset for Pluralistic Alignment
• Abstract summary: Rushes is a dataset and benchmark for studying revealed human engagement preferences in interactive narrative environments.<n>It contains 44,226 decision events from 8,167 unique users across six games.<n>We show that user choices exhibit structured, non-random patterns, quantified by a low choice entropy relative to a uniform baseline.
• Score: 11.198313470367365
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce Rushes, a dataset and benchmark for studying revealed human engagement preferences in interactive narrative environments. Rushes is collected through a game interface where users interact with AI-generated branching narratives and select one choice from a small, explicit candidate set at each decision point. Each interaction logs the full candidate set, the user's choice, and the evolving narrative context, yielding time-ordered trajectories with persistent user-level identifiers. Rushes contains 44,226 decision events from 8,167 unique users across six games, capturing sequential, personalized engagement behavior rather than static judgments. We show that user choices exhibit structured, non-random patterns, quantified by a low choice entropy relative to a uniform baseline. We position Rushes as a diagnostic benchmark for pluralistic alignment and demonstrate a robust Engagement Gap: state-of-the-art LLMs, including GPT-5, fail to outperform simple baselines. While classical Matrix Factorization (SVD) captures measurable personalized signal (37.7%), frontier LLMs (34.23%) struggle to even match the Popularity Baseline (36.4%) on event-level choice prediction. This gap suggests that single, population-level objectives, like those used in modern RLHF, appear insufficient to capture heterogeneous, context-dependent engagement signals. As a result, even highly capable models default to majority preferences rather than adapting to individual trajectories. We release Rushes to support research into pluralistic alignment and sequential decision-making in generative systems. The full code for the platform and dataset will be available here: https://github.com/microsoft/rushes

Related papers

• Low-Resource Preference Adaptation of LLMs via Activation-Based Label Propagation [6.262888209373497]
  We investigate how language models encode preference information in their intermediate representations.<n>We find that activations from chosen and rejected responses form distinct clusters across layers, even in pretrained models.<n>Exploiting this structure, we propose training a lightweight linear probe on a few labelled preference pairs and using it to annotate large unlabelled datasets.
  arXiv  Detail & Related papers  (2026-08-31T14:52:19Z)
• Population-Robust Feature Selection via Generalized Welfare Optimization [14.31847187460321]
  We introduce PopFS, a method for learning one shared, deployable feature set that is robust to population differences.<n>Across eight population splits from six prediction tasks, PopFS consistently achieves strong average and worst-population performance.
  arXiv  Detail & Related papers  (2026-08-03T21:16:54Z)
• Response Time Enhances Alignment with Heterogeneous Preferences [49.69696266152175]
  We show that augmenting preference datasets with a simple, secondary signal can restore identifiability of the population's average preference.<n>Our results bring promises and open up new opportunities for future data-collection pipelines.
  arXiv  Detail & Related papers  (2026-05-07T22:05:23Z)
• Synthetic Interaction Data for Scalable Personalization in Large Language Models [67.31884245564086]
  We introduce a high-fidelity synthetic data generation framework called PersonaGym.<n>Unlike prior work that treats personalization as static persona-preference pairs, PersonaGym models a dynamic preference process.<n>We release PersonaAtlas, a large-scale, high-quality, and diverse synthetic dataset of high-fidelity multi-turn personalized interaction trajectories.
  arXiv  Detail & Related papers  (2026-02-12T20:41:22Z)
• HuggingR$^{4}$: A Progressive Reasoning Framework for Discovering Optimal Model Companions [50.61510609116118]
  HuggingR$4$ is a novel framework that combines Reasoning, Retrieval, Refinement, and Reflection to efficiently select models.<n>It attains a workability rate of 92.03% and a reasonability rate of 82.46%, surpassing existing method by 26.51% and 33.25% respectively.
  arXiv  Detail & Related papers  (2025-11-24T03:13:45Z)
• Robust Preference Alignment via Directional Neighborhood Consensus [13.313830197011983]
  We introduce Robust Preference Selection (RPS), a post-hoc, training-free method by leveraging directional neighborhood consensus.<n>RPS samples multiple responses from a local neighborhood of related preferences to create a superior candidate pool.<n>Our work presents a practical, theoretically-grounded solution for enhancing the reliability of preference-aligned models.
  arXiv  Detail & Related papers  (2025-10-23T12:39:20Z)
• Personalized Reasoning: Just-In-Time Personalization and Why LLMs Fail At It [81.50711040539566]
  Current large language model (LLM) development treats task-solving and preference alignment as separate challenges.<n>We introduce PREFDISCO, an evaluation methodology that transforms static benchmarks into interactive personalization tasks.<n>Our framework creates scenarios where identical questions require different reasoning chains depending on user context.
  arXiv  Detail & Related papers  (2025-09-30T18:55:28Z)
• Few-shot Steerable Alignment: Adapting Rewards and LLM Policies with Neural Processes [50.544186914115045]
  Large language models (LLMs) are increasingly embedded in everyday applications.<n> Ensuring their alignment with the diverse preferences of individual users has become a critical challenge.<n>We present a novel framework for few-shot steerable alignment.
  arXiv  Detail & Related papers  (2024-12-18T16:14:59Z)
• Subjective Crowd Disagreements for Subjective Data: Uncovering Meaningful CrowdOpinion with Population-level Learning [8.530934084017966]
  We introduce emphCrowdOpinion, an unsupervised learning approach that uses language features and label distributions to pool similar items into larger samples of label distributions. We use five publicly available benchmark datasets (with varying levels of annotator disagreements) from social media. We also experiment in the wild using a dataset from Facebook, where annotations come from the platform itself by users reacting to posts.
  arXiv  Detail & Related papers  (2023-07-07T22:09:46Z)
• UniCon: Unified Context Network for Robust Active Speaker Detection [111.90529347692723]
  We introduce a new efficient framework, the Unified Context Network (UniCon), for robust active speaker detection (ASD) Our solution is a novel, unified framework that focuses on jointly modeling multiple types of contextual information. A thorough ablation study is performed on several challenging ASD benchmarks under different settings.
  arXiv  Detail & Related papers  (2021-08-05T13:25:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.