Artificial Generational Intelligence: Cultural Accumulation in Reinforcement Learning

• URL: http://arxiv.org/abs/2406.00392v2
• Date: Mon, 28 Oct 2024 16:33:31 GMT
• Title: Artificial Generational Intelligence: Cultural Accumulation in Reinforcement Learning
• Authors: Jonathan Cook, Chris Lu, Edward Hughes, Joel Z. Leibo, Jakob Foerster,
• Abstract summary: We show that training setups which balance social learning with independent learning give rise to cultural accumulation. In-context and in-weights cultural accumulation can be interpreted as analogous to knowledge and skill accumulation, respectively. This work is the first to present general models that achieve emergent cultural accumulation in reinforcement learning.
• Score: 5.930456214333413
• License:
• Abstract: Cultural accumulation drives the open-ended and diverse progress in capabilities spanning human history. It builds an expanding body of knowledge and skills by combining individual exploration with inter-generational information transmission. Despite its widespread success among humans, the capacity for artificial learning agents to accumulate culture remains under-explored. In particular, approaches to reinforcement learning typically strive for improvements over only a single lifetime. Generational algorithms that do exist fail to capture the open-ended, emergent nature of cultural accumulation, which allows individuals to trade-off innovation and imitation. Building on the previously demonstrated ability for reinforcement learning agents to perform social learning, we find that training setups which balance this with independent learning give rise to cultural accumulation. These accumulating agents outperform those trained for a single lifetime with the same cumulative experience. We explore this accumulation by constructing two models under two distinct notions of a generation: episodic generations, in which accumulation occurs via in-context learning and train-time generations, in which accumulation occurs via in-weights learning. In-context and in-weights cultural accumulation can be interpreted as analogous to knowledge and skill accumulation, respectively. To the best of our knowledge, this work is the first to present general models that achieve emergent cultural accumulation in reinforcement learning, opening up new avenues towards more open-ended learning systems, as well as presenting new opportunities for modelling human culture.

Related papers

• Self-Tuning: Instructing LLMs to Effectively Acquire New Knowledge through Self-Teaching [67.11497198002165]
  Large language models (LLMs) often struggle to provide up-to-date information due to their one-time training. Motivated by the remarkable success of the Feynman Technique in efficient human learning, we introduce Self-Tuning.
  arXiv  Detail & Related papers  (2024-06-10T14:42:20Z)
• Conserving Human Creativity with Evolutionary Generative Algorithms: A Case Study in Music Generation [0.0]
  This study explores the application of evolutionary generative algorithms in music production to preserve and enhance human creativity. By integrating human feedback into Differential Evolution algorithms, we produced six songs that were submitted to international record labels, all of which received contract offers.
  arXiv  Detail & Related papers  (2024-06-09T18:11:05Z)
• Cultural evolution in populations of Large Language Models [15.012901178522874]
  We propose that leveraging the capacity of Large Language Models to mimic human behavior may be fruitful to address this gap. As artificial agents are bound to participate more and more to the evolution of culture, it is crucial to better understand the dynamics of machine-generated cultural evolution. We present a framework for simulating cultural evolution in populations of LLMs, allowing the manipulation of variables known to be important in cultural evolution.
  arXiv  Detail & Related papers  (2024-03-13T18:11:17Z)
• Anti-Retroactive Interference for Lifelong Learning [65.50683752919089]
  We design a paradigm for lifelong learning based on meta-learning and associative mechanism of the brain. It tackles the problem from two aspects: extracting knowledge and memorizing knowledge. It is theoretically analyzed that the proposed learning paradigm can make the models of different tasks converge to the same optimum.
  arXiv  Detail & Related papers  (2022-08-27T09:27:36Z)
• Cumulative culture spontaneously emerges in artificial navigators who are social and memory-guided [0.0]
  Cumulative cultural evolution occurs when adaptive innovations are passed down to consecutive generations through social learning. This process has shaped human technological innovation, but also occurs in non-human species. I show that a much simpler system suffices. Cumulative culture spontaneously emerged in artificial agents who navigate with a minimal cognitive architecture of goal-direction, social proximity, and route memory.
  arXiv  Detail & Related papers  (2022-06-13T16:10:39Z)
• A Unified Continuous Learning Framework for Multi-modal Knowledge Discovery and Pre-training [73.7507857547549]
  We propose to unify knowledge discovery and multi-modal pre-training in a continuous learning framework. For knowledge discovery, a pre-trained model is used to identify cross-modal links on a graph. For model pre-training, the knowledge graph is used as the external knowledge to guide the model updating.
  arXiv  Detail & Related papers  (2022-06-11T16:05:06Z)
• Learning Robust Real-Time Cultural Transmission without Human Data [82.05222093231566]
  We provide a method for generating zero-shot, high recall cultural transmission in artificially intelligent agents. Our agents succeed at real-time cultural transmission from humans in novel contexts without using any pre-collected human data. This paves the way for cultural evolution as an algorithm for developing artificial general intelligence.
  arXiv  Detail & Related papers  (2022-03-01T19:32:27Z)
• Growing knowledge culturally across generations to solve novel, complex tasks [29.579223105173217]
  We take a first step towards reverse-engineering cultural learning through language. We develop a suite of complex high-stakes tasks in the form of minimalist-style video games. Knowledge accumulated gradually across generations, allowing later generations to advance further in the games.
  arXiv  Detail & Related papers  (2021-07-28T14:09:40Z)
• Emergent Social Learning via Multi-agent Reinforcement Learning [91.57176641192771]
  Social learning is a key component of human and animal intelligence. This paper investigates whether independent reinforcement learning agents can learn to use social learning to improve their performance.
  arXiv  Detail & Related papers  (2020-10-01T17:54:14Z)
• Language Generation with Multi-Hop Reasoning on Commonsense Knowledge Graph [124.45799297285083]
  We argue that exploiting both the structural and semantic information of the knowledge graph facilitates commonsense-aware text generation. We propose Generation with Multi-Hop Reasoning Flow (GRF) that enables pre-trained models with dynamic multi-hop reasoning on multi-relational paths extracted from the external commonsense knowledge graph.
  arXiv  Detail & Related papers  (2020-09-24T13:55:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.