Meta-control of social learning strategies

• URL: http://arxiv.org/abs/2106.10015v1
• Date: Fri, 18 Jun 2021 09:17:21 GMT
• Title: Meta-control of social learning strategies
• Authors: Anil Yaman, Nicolas Bredeche, Onur \c{C}aylak, Joel Z. Leibo, Sang Wan Lee
• Abstract summary: Social learning, copying other's behavior without actual experience, offers a cost-effective means of knowledge acquisition. We show here that while the success-based strategy fully exploits the benign environment of low uncertainly, it fails in uncertain environments. On the other hand, the conformist strategy can effectively mitigate this adverse effect.
• Score: 9.419484512715242
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Social learning, copying other's behavior without actual experience, offers a cost-effective means of knowledge acquisition. However, it raises the fundamental question of which individuals have reliable information: successful individuals versus the majority. The former and the latter are known respectively as success-based and conformist social learning strategies. We show here that while the success-based strategy fully exploits the benign environment of low uncertainly, it fails in uncertain environments. On the other hand, the conformist strategy can effectively mitigate this adverse effect. Based on these findings, we hypothesized that meta-control of individual and social learning strategies provides effective and sample-efficient learning in volatile and uncertain environments. Simulations on a set of environments with various levels of volatility and uncertainty confirmed our hypothesis. The results imply that meta-control of social learning affords agents the leverage to resolve environmental uncertainty with minimal exploration cost, by exploiting others' learning as an external knowledge base.

Related papers

• MARO: Learning Stronger Reasoning from Social Interaction [7.77506109184819]
  Multi-Agent Reward Optimization (MARO) is a method that enables large language models to acquire stronger reasoning abilities.<n> Experimental results demonstrate that MARO achieves significant improvements in social reasoning capabilities.
  arXiv  Detail & Related papers  (2026-01-18T09:10:08Z)
• Autonomous Learning From Success and Failure: Goal-Conditioned Supervised Learning with Negative Feedback [2.36462256498849]
  Goal-Conditioned Supervised Learning has emerged as a potential solution by enabling self-imitation learning for autonomous systems.<n>We propose a novel model that integrates contrastive learning principles into the GCSL framework to learn from both success and failure.
  arXiv  Detail & Related papers  (2025-09-03T10:50:48Z)
• The Sample Complexity of Online Strategic Decision Making with Information Asymmetry and Knowledge Transportability [93.11220429350278]
  Information asymmetry is a pervasive feature of multi-agent systems.<n>This paper explores a fundamental question in online learning: Can we employ non-i.i.d. actions to learn about confounders even when requiring knowledge transfer?<n>We present a sample-efficient algorithm designed to accurately identify system dynamics under information asymmetry and to navigate the challenges of knowledge transfer effectively in reinforcement learning.
  arXiv  Detail & Related papers  (2025-06-11T17:06:57Z)
• Information-Seeking Decision Strategies Mitigate Risk in Dynamic, Uncertain Environments [0.0]
  We compare the performance of normative reward- and information-seeking strategies in a foraging task. We find subtle disparities in the actions they take, resulting in meaningful performance differences. Our findings support the adaptive value of information-seeking behaviors that can mitigate risk with minimal reward loss.
  arXiv  Detail & Related papers  (2025-03-24T19:55:41Z)
• Mind the (Belief) Gap: Group Identity in the World of LLMs [22.96432452893247]
  Social biases and belief-driven behaviors can significantly impact Large Language Models (LLMs) decisions on several tasks. We present a multi-agent framework that simulates belief congruence, a classical group psychology theory that plays a crucial role in shaping societal interactions and preferences.
  arXiv  Detail & Related papers  (2025-03-03T19:50:52Z)
• Markets with Heterogeneous Agents: Dynamics and Survival of Bayesian vs. No-Regret Learners [3.985264439635754]
  We analyze the performance of heterogeneous learning agents in asset markets with payoffs.<n>Surprisingly, we find that low regret is not sufficient for survival.<n>No-regret learning requires less knowledge of the environment and is therefore more robust.
  arXiv  Detail & Related papers  (2025-02-12T17:34:04Z)
• Deviations from the Nash equilibrium and emergence of tacit collusion in a two-player optimal execution game with reinforcement learning [0.9208007322096533]
  We study a scenario in which two autonomous agents learn to liquidate the same asset optimally in the presence of market impact. Our results show that the strategies learned by the agents deviate significantly from the Nash equilibrium of the corresponding market impact game. We explore how different levels of market volatility influence the agents' performance and the equilibria they discover.
  arXiv  Detail & Related papers  (2024-08-21T16:54:53Z)
• Uncertainty for Active Learning on Graphs [70.44714133412592]
  Uncertainty Sampling is an Active Learning strategy that aims to improve the data efficiency of machine learning models. We benchmark Uncertainty Sampling beyond predictive uncertainty and highlight a significant performance gap to other Active Learning strategies. We develop ground-truth Bayesian uncertainty estimates in terms of the data generating process and prove their effectiveness in guiding Uncertainty Sampling toward optimal queries.
  arXiv  Detail & Related papers  (2024-05-02T16:50:47Z)
• Assessor-Guided Learning for Continual Environments [17.181933166255448]
  This paper proposes an assessor-guided learning strategy for continual learning. An assessor guides the learning process of a base learner by controlling the direction and pace of the learning process. The assessor is trained in a meta-learning manner with a meta-objective to boost the learning process of the base learner.
  arXiv  Detail & Related papers  (2023-03-21T06:45:14Z)
• Bridging adaptive management and reinforcement learning for more robust decisions [6.152873761869356]
  We show how reinforcement learning can help us devise robust strategies for managing environmental systems under great uncertainty. Our synthesis suggests that environmental management and computer science can learn from one another about the practices, promises, and perils of experience-based decision-making.
  arXiv  Detail & Related papers  (2023-03-15T16:14:12Z)
• Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarios [147.16925581385576]
  We show how imitation learning combined with reinforcement learning can substantially improve the safety and reliability of driving policies. We train a policy on over 100k miles of urban driving data, and measure its effectiveness in test scenarios grouped by different levels of collision likelihood.
  arXiv  Detail & Related papers  (2022-12-21T23:59:33Z)
• Flexible social inference facilitates targeted social learning when rewards are not observable [58.762004496858836]
  Groups coordinate more effectively when individuals are able to learn from others' successes. We suggest that social inference capacities may help bridge this gap, allowing individuals to update their beliefs about others' underlying knowledge and success from observable trajectories of behavior.
  arXiv  Detail & Related papers  (2022-12-01T21:04:03Z)
• Imitating Past Successes can be Very Suboptimal [145.70788608016755]
  We show that existing outcome-conditioned imitation learning methods do not necessarily improve the policy. We show that a simple modification results in a method that does guarantee policy improvement. Our aim is not to develop an entirely new method, but rather to explain how a variant of outcome-conditioned imitation learning can be used to maximize rewards.
  arXiv  Detail & Related papers  (2022-06-07T15:13:43Z)
• Social learning spontaneously emerges by searching optimal heuristics with deep reinforcement learning [0.0]
  We employ a deep reinforcement learning model to optimize the social learning strategies of agents in a cooperative game in a multi-dimensional landscape. We find that the agent spontaneously learns various concepts of social learning, such as copying, focusing on frequent and well-performing neighbors, self-comparison, and the importance of balancing between individual and social learning. We demonstrate the superior performance of the reinforcement learning agent in various environments, including temporally changing environments and real social networks.
  arXiv  Detail & Related papers  (2022-04-26T15:10:27Z)
• Disturbing Reinforcement Learning Agents with Corrupted Rewards [62.997667081978825]
  We analyze the effects of different attack strategies based on reward perturbations on reinforcement learning algorithms. We show that smoothly crafting adversarial rewards are able to mislead the learner, and that using low exploration probability values, the policy learned is more robust to corrupt rewards.
  arXiv  Detail & Related papers  (2021-02-12T15:53:48Z)
• Emergent Social Learning via Multi-agent Reinforcement Learning [91.57176641192771]
  Social learning is a key component of human and animal intelligence. This paper investigates whether independent reinforcement learning agents can learn to use social learning to improve their performance.
  arXiv  Detail & Related papers  (2020-10-01T17:54:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.