Language Model Goal Selection Differs from Humans' in an Open-Ended Task

• URL: http://arxiv.org/abs/2603.03295v1
• Date: Fri, 06 Feb 2026 15:39:54 GMT
• Title: Language Model Goal Selection Differs from Humans' in an Open-Ended Task
• Abstract summary: Large language models (LLMs) are increasingly integrated into human decision-making.<n>We assess the validity of LLMs as proxies for human goal selection in a controlled, open-ended learning task borrowed from cognitive science.
• Score: 2.012425476229879
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As large language models (LLMs) get integrated into human decision-making, they are increasingly choosing goals autonomously rather than only completing human-defined ones, assuming they will reflect human preferences. However, human-LLM similarity in goal selection remains largely untested. We directly assess the validity of LLMs as proxies for human goal selection in a controlled, open-ended learning task borrowed from cognitive science. Across four state-of-the-art models (GPT-5, Gemini 2.5 Pro, Claude Sonnet 4.5, and Centaur), we find substantial divergence from human behavior. While people gradually explore and learn to achieve goals with diversity across individuals, most models exploit a single identified solution (reward hacking) or show surprisingly low performance, with distinct patterns across models and little variability across instances of the same model. Even Centaur, explicitly trained to emulate humans in experimental settings, poorly captures people's goal selection. Chain-of-thought reasoning and persona steering provide limited improvements. These findings highlight the uniqueness of human goal selection, cautioning against replacing it with current models in applications such as personal assistance, scientific discovery, and policy research.

Related papers

• Aspire: Can Models Self-Evolve from Vague Goals? [61.65007006747275]
  We introduce ASPIRE, a benchmark for vague-goal-driven self-evolution.<n>ASPIRE supports both model-weight and agent-harness evolution in a unified interactive environment.<n>Our experiments show that vague goals redirect search effort toward goal interpretation.
  arXiv  Detail & Related papers  (2026-08-31T17:14:59Z)
• Matched Outcomes, Divergent Gaze: How Foveated MLLMs Search Compared to Humans [50.71222402701153]
  We compare three general-purpose large language models (MLLMs) with human eye-movement scanpaths on goal-directed search (COCO-Search18)<n>All three share one signature: low-entropy, large-amplitude, self-consistent scanpaths that agree with themselves far more closely than two humans agree with each other.<n>The gap sits on a process axis that answer-alignment and saliency metrics do not measure. Because they miss it, such metrics cannot certify human-like vision, and zero-shot models suit outcome and spatial questions but not temporal, process-level
  arXiv  Detail & Related papers  (2026-08-17T12:54:39Z)
• HumanLLM: Towards Personalized Understanding and Simulation of Human Nature [72.55730315685837]
  HumanLLM is a foundation model designed for personalized understanding and simulation of individuals.<n>We first construct the Cognitive Genome, a large-scale corpus curated from real-world user data on platforms like Reddit, Twitter, Blogger, and Amazon.<n>We then formulate diverse learning tasks and perform supervised fine-tuning to empower the model to predict a wide range of individualized human behaviors, thoughts, and experiences.
  arXiv  Detail & Related papers  (2026-01-22T09:27:27Z)
• Learning to Imitate with Less: Efficient Individual Behavior Modeling in Chess [10.090379544417432]
  Maia4All is a framework designed to learn and adapt to individual decision-making styles efficiently.<n>Maia4All achieves individual human behavior modeling in chess with only 20 games, compared to the 5,000 games required previously.
  arXiv  Detail & Related papers  (2025-07-29T04:09:31Z)
• Empirically evaluating commonsense intelligence in large language models with large-scale human judgments [4.212429064310439]
  We propose a method for evaluating common sense in artificial intelligence.<n>We measure the correspondence between a model's judgment and that of a human population.<n>Our framework contributes to the growing call for adapting AI models to human collectivities that possess different, often incompatible, social stocks of knowledge.
  arXiv  Detail & Related papers  (2025-05-15T13:55:27Z)
• Towards Neural Network based Cognitive Models of Dynamic Decision-Making by Humans [16.72938921687168]
  We build on Instance Based Learning (IBL) that posits that human decisions are based on similar situations encountered in the past. We propose two new attention-based neural network models to have open form non-linear functions to model distinct and heterogeneous human decision-making.
  arXiv  Detail & Related papers  (2024-07-24T20:28:03Z)
• MaxMin-RLHF: Alignment with Diverse Human Preferences [101.57443597426374]
  Reinforcement Learning from Human Feedback (RLHF) aligns language models to human preferences by employing a singular reward model derived from preference data.<n>We learn a mixture of preference distributions via an expectation-maximization algorithm to better represent diverse human preferences.<n>Our algorithm achieves an average improvement of more than 16% in win-rates over conventional RLHF algorithms.
  arXiv  Detail & Related papers  (2024-02-14T03:56:27Z)
• Promptable Behaviors: Personalizing Multi-Objective Rewards from Human Preferences [53.353022588751585]
  We present Promptable Behaviors, a novel framework that facilitates efficient personalization of robotic agents to diverse human preferences. We introduce three distinct methods to infer human preferences by leveraging different types of interactions. We evaluate the proposed method in personalized object-goal navigation and flee navigation tasks in ProcTHOR and RoboTHOR.
  arXiv  Detail & Related papers  (2023-12-14T21:00:56Z)
• Learning signatures of decision making from many individuals playing the same game [54.33783158658077]
  We design a predictive framework that learns representations to encode an individual's 'behavioral style' We apply our method to a large-scale behavioral dataset from 1,000 humans playing a 3-armed bandit task.
  arXiv  Detail & Related papers  (2023-02-21T21:41:53Z)
• Humans are not Boltzmann Distributions: Challenges and Opportunities for Modelling Human Feedback and Interaction in Reinforcement Learning [13.64577704565643]
  We argue that these models are too simplistic and that RL researchers need to develop more realistic human models to design and evaluate their algorithms. This paper calls for research from different disciplines to address key questions about how humans provide feedback to AIs and how we can build more robust human-in-the-loop RL systems.
  arXiv  Detail & Related papers  (2022-06-27T13:58:51Z)
• Weak Human Preference Supervision For Deep Reinforcement Learning [48.03929962249475]
  The current reward learning from human preferences could be used to resolve complex reinforcement learning (RL) tasks without access to a reward function. We propose a weak human preference supervision framework, for which we developed a human preference scaling model. Our established human-demonstration estimator requires human feedback only for less than 0.01% of the agent's interactions with the environment.
  arXiv  Detail & Related papers  (2020-07-25T10:37:15Z)
• LESS is More: Rethinking Probabilistic Models of Human Behavior [36.020541093946925]
  Boltzmann noisily-rational decision model assumes people approximately optimize a reward function. Human trajectories lie in a continuous space, with continuous-valued features that influence the reward function. We introduce a model that explicitly accounts for distances between trajectories, rather than only their rewards.
  arXiv  Detail & Related papers  (2020-01-13T18:59:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.