Mindstorms in Natural Language-Based Societies of Mind

• URL: http://arxiv.org/abs/2305.17066v1
• Date: Fri, 26 May 2023 16:21:25 GMT
• Title: Mindstorms in Natural Language-Based Societies of Mind
• Authors: Mingchen Zhuge, Haozhe Liu, Francesco Faccio, Dylan R. Ashley, R\'obert Csord\'as, Anand Gopalakrishnan, Abdullah Hamdi, Hasan Abed Al Kader Hammoud, Vincent Herrmann, Kazuki Irie, Louis Kirsch, Bing Li, Guohao Li, Shuming Liu, Jinjie Mai, Piotr Pi\k{e}kos, Aditya Ramesh, Imanol Schlag, Weimin Shi, Aleksandar Stani\'c, Wenyi Wang, Yuhui Wang, Mengmeng Xu, Deng-Ping Fan, Bernard Ghanem, J\"urgen Schmidhuber
• Abstract summary: Minsky's "society of mind" and Schmidhuber's "learning to think" inspire diverse societies of large multimodal neural networks (NNs) Recent implementations of NN-based societies of minds consist of large language models (LLMs) and other NN-based experts communicating through a natural language interface. In these natural language-based societies of mind (NLSOMs), new agents -- all communicating through the same universal symbolic language -- are easily added in a modular fashion.
• Score: 110.05229611910478
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Both Minsky's "society of mind" and Schmidhuber's "learning to think" inspire diverse societies of large multimodal neural networks (NNs) that solve problems by interviewing each other in a "mindstorm." Recent implementations of NN-based societies of minds consist of large language models (LLMs) and other NN-based experts communicating through a natural language interface. In doing so, they overcome the limitations of single LLMs, improving multimodal zero-shot reasoning. In these natural language-based societies of mind (NLSOMs), new agents -- all communicating through the same universal symbolic language -- are easily added in a modular fashion. To demonstrate the power of NLSOMs, we assemble and experiment with several of them (having up to 129 members), leveraging mindstorms in them to solve some practical AI tasks: visual question answering, image captioning, text-to-image synthesis, 3D generation, egocentric retrieval, embodied AI, and general language-based task solving. We view this as a starting point towards much larger NLSOMs with billions of agents-some of which may be humans. And with this emergence of great societies of heterogeneous minds, many new research questions have suddenly become paramount to the future of artificial intelligence. What should be the social structure of an NLSOM? What would be the (dis)advantages of having a monarchical rather than a democratic structure? How can principles of NN economies be used to maximize the total reward of a reinforcement learning NLSOM? In this work, we identify, discuss, and try to answer some of these questions.

Related papers

• From Frege to chatGPT: Compositionality in language, cognition, and deep neural networks [0.0]
  We review recent empirical work from machine learning for a broad audience in philosophy, cognitive science, and neuroscience. In particular, our review emphasizes two approaches to endowing neural networks with compositional generalization capabilities. We conclude by discussing the implications that these findings may have for the study of compositionality in human cognition.
  arXiv  Detail & Related papers  (2024-05-24T02:36:07Z)
• Sphere Neural-Networks for Rational Reasoning [24.591858077975548]
  We present a novel extension by generalising computational building blocks from vectors to spheres. We propose Sphere Neural Networks (SphNNs) for human-like reasoning through model construction and inspection. SphNNs can evolve into various types of reasoning, such as rationality-temporal reasoning, logical reasoning with negation and disjunction, event reasoning, neuro-symbolic unification, and humour understanding.
  arXiv  Detail & Related papers  (2024-03-22T15:44:59Z)
• Brain in a Vat: On Missing Pieces Towards Artificial General Intelligence in Large Language Models [83.63242931107638]
  We propose four characteristics of generally intelligent agents. We argue that active engagement with objects in the real world delivers more robust signals for forming conceptual representations. We conclude by outlining promising future research directions in the field of artificial general intelligence.
  arXiv  Detail & Related papers  (2023-07-07T13:58:16Z)
• Understanding Natural Language Understanding Systems. A Critical Analysis [91.81211519327161]
  The development of machines that guillemotlefttalk like usguillemotright, also known as Natural Language Understanding (NLU) systems, is the Holy Grail of Artificial Intelligence (AI) But never has the trust that we can build guillemotlefttalking machinesguillemotright been stronger than the one engendered by the last generation of NLU systems. Are we at the dawn of a new era, in which the Grail is finally closer to us?
  arXiv  Detail & Related papers  (2023-03-01T08:32:55Z)
• Communication Drives the Emergence of Language Universals in Neural Agents: Evidence from the Word-order/Case-marking Trade-off [3.631024220680066]
  We propose a new Neural-agent Language Learning and Communication framework (NeLLCom) where pairs of speaking and listening agents first learn a miniature language. We succeed in replicating the trade-off with the new framework without hard-coding specific biases in the agents.
  arXiv  Detail & Related papers  (2023-01-30T17:22:33Z)
• Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMs [77.88043871260466]
  We show that one of today's largest language models lacks this kind of social intelligence out-of-the box. We conclude that person-centric NLP approaches might be more effective towards neural Theory of Mind.
  arXiv  Detail & Related papers  (2022-10-24T14:58:58Z)
• Human-guided Collaborative Problem Solving: A Natural Language based Framework [74.27063862727849]
  Our framework consists of three components -- a natural language engine that parses the language utterances to a formal representation and vice-versa. We illustrate the ability of this framework to address the key challenges of collaborative problem solving by demonstrating it on a collaborative building task in a Minecraft-based blocksworld domain.
  arXiv  Detail & Related papers  (2022-07-19T21:52:37Z)
• Emergence of Machine Language: Towards Symbolic Intelligence with Neural Networks [73.94290462239061]
  We propose to combine symbolism and connectionism principles by using neural networks to derive a discrete representation. By designing an interactive environment and task, we demonstrated that machines could generate a spontaneous, flexible, and semantic language.
  arXiv  Detail & Related papers  (2022-01-14T14:54:58Z)
• SocialAI 0.1: Towards a Benchmark to Stimulate Research on Socio-Cognitive Abilities in Deep Reinforcement Learning Agents [23.719833581321033]
  Building embodied autonomous agents capable of participating in social interactions with humans is one of the main challenges in AI. Current approaches focus on language as a communication tool in very simplified and non diverse social situations. We argue that aiming towards human-level AI requires a broader set of key social skills.
  arXiv  Detail & Related papers  (2021-04-27T14:16:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.