Have I done enough planning or should I plan more?

• URL: http://arxiv.org/abs/2201.00764v1
• Date: Mon, 3 Jan 2022 17:11:07 GMT
• Title: Have I done enough planning or should I plan more?
• Authors: Ruiqi He, Yash Raj Jain, Falk Lieder
• Abstract summary: We show that people acquire this ability through learning and reverse-engineer the underlying learning mechanisms. We find that people quickly adapt how much planning they perform to the cost and benefit of planning. Our results suggest that the metacognitive ability to adjust the amount of planning might be learned through a policy-gradient mechanism.
• Score: 0.7734726150561086
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: People's decisions about how to allocate their limited computational resources are essential to human intelligence. An important component of this metacognitive ability is deciding whether to continue thinking about what to do and move on to the next decision. Here, we show that people acquire this ability through learning and reverse-engineer the underlying learning mechanisms. Using a process-tracing paradigm that externalises human planning, we find that people quickly adapt how much planning they perform to the cost and benefit of planning. To discover the underlying metacognitive learning mechanisms we augmented a set of reinforcement learning models with metacognitive features and performed Bayesian model selection. Our results suggest that the metacognitive ability to adjust the amount of planning might be learned through a policy-gradient mechanism that is guided by metacognitive pseudo-rewards that communicate the value of planning.

Related papers

• SPIRE: Synergistic Planning, Imitation, and Reinforcement Learning for Long-Horizon Manipulation [58.14969377419633]
  We propose spire, a system that decomposes tasks into smaller learning subproblems and second combines imitation and reinforcement learning to maximize their strengths. We find that spire outperforms prior approaches that integrate imitation learning, reinforcement learning, and planning by 35% to 50% in average task performance.
  arXiv  Detail & Related papers  (2024-10-23T17:42:07Z)
• Towards Generalizable Zero-Shot Manipulation via Translating Human Interaction Plans [58.27029676638521]
  We show how passive human videos can serve as a rich source of data for learning such generalist robots. We learn a human plan predictor that, given a current image of a scene and a goal image, predicts the future hand and object configurations. We show that our learned system can perform over 16 manipulation skills that generalize to 40 objects.
  arXiv  Detail & Related papers  (2023-12-01T18:54:12Z)
• What are the mechanisms underlying metacognitive learning? [5.787117733071415]
  We postulate that people learn this ability from trial and error (metacognitive reinforcement learning) Here, we systematize models of the underlying learning mechanisms and enhance them with more sophisticated additional mechanisms. Our results suggest that a gradient ascent through the space of cognitive strategies can explain most of the observed qualitative phenomena.
  arXiv  Detail & Related papers  (2023-02-09T18:49:10Z)
• An intelligent tutor for planning in large partially observable environments [0.8739101659113157]
  We develop and evaluate the first intelligent tutor for planning in partially observable environments. Compared to previous intelligent tutors for teaching planning strategies, this novel intelligent tutor combines two innovations. A preregistered experiment with 330 participants demonstrated that the new intelligent tutor is highly effective at improving people's ability to make good decisions in partially observable environments.
  arXiv  Detail & Related papers  (2023-02-06T13:57:08Z)
• Achieving mouse-level strategic evasion performance using real-time computational planning [59.60094442546867]
  Planning is an extraordinary ability in which the brain imagines and then enacts evaluated possible futures. We develop a more efficient biologically-inspired planning algorithm, TLPPO, based on work on how the ecology of an animal governs the value of spatial planning. We compare the performance of a real-time agent using TLPPO against the performance of live mice, all tasked with evading a robot predator.
  arXiv  Detail & Related papers  (2022-11-04T18:34:36Z)
• Anti-Retroactive Interference for Lifelong Learning [65.50683752919089]
  We design a paradigm for lifelong learning based on meta-learning and associative mechanism of the brain. It tackles the problem from two aspects: extracting knowledge and memorizing knowledge. It is theoretically analyzed that the proposed learning paradigm can make the models of different tasks converge to the same optimum.
  arXiv  Detail & Related papers  (2022-08-27T09:27:36Z)
• Procedure Planning in Instructional Videosvia Contextual Modeling and Model-based Policy Learning [114.1830997893756]
  This work focuses on learning a model to plan goal-directed actions in real-life videos. We propose novel algorithms to model human behaviors through Bayesian Inference and model-based Imitation Learning.
  arXiv  Detail & Related papers  (2021-10-05T01:06:53Z)
• Automatic discovery and description of human planning strategies [0.7734726150561086]
  We leverage AI for strategy discovery for understanding human planning. Our algorithm, called Human-Interpret, uses imitation learning to describe process-tracing data. We find that the descriptions of human planning strategies obtained automatically are about as understandable as human-generated descriptions.
  arXiv  Detail & Related papers  (2021-09-29T15:20:16Z)
• Improving Human Decision-Making by Discovering Efficient Strategies for Hierarchical Planning [0.6882042556551609]
  People need efficient planning strategies because their computational resources are limited. Our ability to compute those strategies used to be limited to very small and very simple planning tasks. We introduce a cognitively-inspired reinforcement learning method that can overcome this limitation.
  arXiv  Detail & Related papers  (2021-01-31T19:46:00Z)
• The Efficiency of Human Cognition Reflects Planned Information Processing [40.51474966524166]
  We make predictions about how people should plan and meta-plan as a function of the overall structure of a task. We find that people's reaction times reflect a planned use of information processing. This formulation of planning to plan provides new insight into the function of hierarchical planning, state abstraction, and cognitive control in both humans and machines.
  arXiv  Detail & Related papers  (2020-02-13T20:34:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.