Individual differences in the cognitive mechanisms of planning strategy discovery

• URL: http://arxiv.org/abs/2505.23519v1
• Date: Thu, 29 May 2025 14:57:34 GMT
• Title: Individual differences in the cognitive mechanisms of planning strategy discovery
• Authors: Ruiqi He, Falk Lieder,
• Abstract summary: We investigate whether incorporating cognitive mechanisms that might facilitate human strategy discovery can bring models of metacognitive reinforcement learning closer to human performance.<n>We consider intrinsically generated metacognitive pseudo-rewards, subjective effort valuation, and termination deliberation.<n>While these enhancements provided valuable insights into individual differences and the effect of these mechanisms on strategy discovery, they did not fully close the gap between model and human performance.
• Score: 0.9821874476902969
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: People employ efficient planning strategies. But how are these strategies acquired? Previous research suggests that people can discover new planning strategies through learning from reinforcements, a process known as metacognitive reinforcement learning (MCRL). While prior work has shown that MCRL models can learn new planning strategies and explain more participants' experience-driven discovery better than alternative mechanisms, it also revealed significant individual differences in metacognitive learning. Furthermore, when fitted to human data, these models exhibit a slower rate of strategy discovery than humans. In this study, we investigate whether incorporating cognitive mechanisms that might facilitate human strategy discovery can bring models of MCRL closer to human performance. Specifically, we consider intrinsically generated metacognitive pseudo-rewards, subjective effort valuation, and termination deliberation. Analysis of planning task data shows that a larger proportion of participants used at least one of these mechanisms, with significant individual differences in their usage and varying impacts on strategy discovery. Metacognitive pseudo-rewards, subjective effort valuation, and learning the value of acting without further planning were found to facilitate strategy discovery. While these enhancements provided valuable insights into individual differences and the effect of these mechanisms on strategy discovery, they did not fully close the gap between model and human performance, prompting further exploration of additional factors that people might use to discover new planning strategies.

Related papers

• MiMu: Mitigating Multiple Shortcut Learning Behavior of Transformers [19.27328009299697]
  Empirical Risk Minimization (ERM) models often rely on spurious correlations between features and labels during the learning process.<n>We propose MiMu, a novel method integrated with Transformer-based generalizations designed to Mitigate Multiple shortcut learning behavior.
  arXiv  Detail & Related papers  (2025-04-14T08:11:09Z)
• Experience-driven discovery of planning strategies [0.9821874476902969]
  We show that new planning strategies are discovered through metacognitive reinforcement learning.<n>When fitted to human data, these models exhibit a slower discovery rate than humans, leaving room for improvement.
  arXiv  Detail & Related papers  (2024-12-04T08:20:03Z)
• Towards a Unified View of Preference Learning for Large Language Models: A Survey [88.66719962576005]
  Large Language Models (LLMs) exhibit remarkably powerful capabilities. One of the crucial factors to achieve success is aligning the LLM's output with human preferences. We decompose all the strategies in preference learning into four components: model, data, feedback, and algorithm.
  arXiv  Detail & Related papers  (2024-09-04T15:11:55Z)
• LLM as a Mastermind: A Survey of Strategic Reasoning with Large Language Models [75.89014602596673]
  Strategic reasoning requires understanding and predicting adversary actions in multi-agent settings while adjusting strategies accordingly. We explore the scopes, applications, methodologies, and evaluation metrics related to strategic reasoning with Large Language Models. It underscores the importance of strategic reasoning as a critical cognitive capability and offers insights into future research directions and potential improvements.
  arXiv  Detail & Related papers  (2024-04-01T16:50:54Z)
• Enhancing Generative Class Incremental Learning Performance with Model Forgetting Approach [50.36650300087987]
  This study presents a novel approach to Generative Class Incremental Learning (GCIL) by introducing the forgetting mechanism. We have found that integrating the forgetting mechanisms significantly enhances the models' performance in acquiring new knowledge.
  arXiv  Detail & Related papers  (2024-03-27T05:10:38Z)
• Inducing Individual Students' Learning Strategies through Homomorphic POMDPs [3.388937792941359]
  We propose the homomorphic POMDP (H-POMDP) model to accommodate multiple cognitive patterns. Based on the H-POMDP model, we are able to represent different cognitive patterns from the data. We conduct experiments to show that, in comparison to the general POMDP approach, the H-POMDP model demonstrates better precision when modelling mixed data from multiple cognitive patterns.
  arXiv  Detail & Related papers  (2024-03-16T14:06:29Z)
• Program-Based Strategy Induction for Reinforcement Learning [5.657991642023959]
  We use Bayesian program induction to discover strategies implemented by programs, letting the simplicity of strategies trade off against their effectiveness. We find strategies that are difficult or unexpected with classical incremental learning, like asymmetric learning from rewarded and unrewarded trials, adaptive horizon-dependent random exploration, and discrete state switching.
  arXiv  Detail & Related papers  (2024-02-26T15:40:46Z)
• Evolving Strategies for Competitive Multi-Agent Search [12.699427247517031]
  This article first formalizes human creative problem solving as competitive multi-agent search (CMAS) The main hypothesis is that evolutionary computation can be used to discover effective strategies for CMAS. Different specialized strategies are evolved for each different competitive environment, and also general strategies that perform well across environments.
  arXiv  Detail & Related papers  (2023-06-18T21:35:41Z)
• Machine Psychology [54.287802134327485]
  We argue that a fruitful direction for research is engaging large language models in behavioral experiments inspired by psychology. We highlight theoretical perspectives, experimental paradigms, and computational analysis techniques that this approach brings to the table. It paves the way for a "machine psychology" for generative artificial intelligence (AI) that goes beyond performance benchmarks.
  arXiv  Detail & Related papers  (2023-03-24T13:24:41Z)
• Anti-Retroactive Interference for Lifelong Learning [65.50683752919089]
  We design a paradigm for lifelong learning based on meta-learning and associative mechanism of the brain. It tackles the problem from two aspects: extracting knowledge and memorizing knowledge. It is theoretically analyzed that the proposed learning paradigm can make the models of different tasks converge to the same optimum.
  arXiv  Detail & Related papers  (2022-08-27T09:27:36Z)
• Insights into Data through Model Behaviour: An Explainability-driven Strategy for Data Auditing for Responsible Computer Vision Applications [70.92379567261304]
  This study explores an explainability-driven strategy to data auditing. We demonstrate this strategy by auditing two popular medical benchmark datasets. We discover hidden data quality issues that lead deep learning models to make predictions for the wrong reasons.
  arXiv  Detail & Related papers  (2021-06-16T23:46:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.