Symmetry-Based Representations for Artificial and Biological General Intelligence

• URL: http://arxiv.org/abs/2203.09250v1
• Date: Thu, 17 Mar 2022 11:18:34 GMT
• Title: Symmetry-Based Representations for Artificial and Biological General Intelligence
• Authors: Irina Higgins, S\'ebastien Racani\`ere, Danilo Rezende
• Abstract summary: We argue that symmetry transformations are a fundamental principle that can guide our search for what makes a good representation. symmetries have started to gain prominence in machine learning too, resulting in more data efficient and generalisable algorithms. First demonstrations of the importance of symmetry transformations for representation learning in the brain are starting to arise in neuroscience.
• Score: 4.39338211982718
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Biological intelligence is remarkable in its ability to produce complex behaviour in many diverse situations through data efficient, generalisable and transferable skill acquisition. It is believed that learning "good" sensory representations is important for enabling this, however there is little agreement as to what a good representation should look like. In this review article we are going to argue that symmetry transformations are a fundamental principle that can guide our search for what makes a good representation. The idea that there exist transformations (symmetries) that affect some aspects of the system but not others, and their relationship to conserved quantities has become central in modern physics, resulting in a more unified theoretical framework and even ability to predict the existence of new particles. Recently, symmetries have started to gain prominence in machine learning too, resulting in more data efficient and generalisable algorithms that can mimic some of the complex behaviours produced by biological intelligence. Finally, first demonstrations of the importance of symmetry transformations for representation learning in the brain are starting to arise in neuroscience. Taken together, the overwhelming positive effect that symmetries bring to these disciplines suggest that they may be an important general framework that determines the structure of the universe, constrains the nature of natural tasks and consequently shapes both biological and artificial intelligence.

Related papers

• Artificial Kuramoto Oscillatory Neurons [65.16453738828672]
  We introduce Artificial Kuramotoy Neurons (AKOrN) as a dynamical alternative to threshold units. We show that this idea provides performance improvements across a wide spectrum of tasks. We believe that these empirical results show the importance of our assumptions at the most basic neuronal level of neural representation.
  arXiv  Detail & Related papers  (2024-10-17T17:47:54Z)
• Binding Dynamics in Rotating Features [72.80071820194273]
  We propose an alternative "cosine binding" mechanism, which explicitly computes the alignment between features and adjusts weights accordingly. This allows us to draw direct connections to self-attention and biological neural processes, and to shed light on the fundamental dynamics for object-centric representations to emerge in Rotating Features.
  arXiv  Detail & Related papers  (2024-02-08T12:31:08Z)
• Exploring mechanisms of Neural Robustness: probing the bridge between geometry and spectrum [0.0]
  We study the link between representation smoothness and spectrum by using weight, Jacobian and spectral regularization. Our research aims to understand the interplay between geometry, spectral properties, robustness, and expressivity in neural representations.
  arXiv  Detail & Related papers  (2024-02-05T12:06:00Z)
• Brain-Inspired Machine Intelligence: A Survey of Neurobiologically-Plausible Credit Assignment [65.268245109828]
  We examine algorithms for conducting credit assignment in artificial neural networks that are inspired or motivated by neurobiology. We organize the ever-growing set of brain-inspired learning schemes into six general families and consider these in the context of backpropagation of errors. The results of this review are meant to encourage future developments in neuro-mimetic systems and their constituent learning processes.
  arXiv  Detail & Related papers  (2023-12-01T05:20:57Z)
• A Neuro-mimetic Realization of the Common Model of Cognition via Hebbian Learning and Free Energy Minimization [55.11642177631929]
  Large neural generative models are capable of synthesizing semantically rich passages of text or producing complex images. We discuss the COGnitive Neural GENerative system, such an architecture that casts the Common Model of Cognition.
  arXiv  Detail & Related papers  (2023-10-14T23:28:48Z)
• Symmetry and Complexity in Object-Centric Deep Active Inference Models [4.298360054690217]
  We show how inherent symmetries of particular objects emerge as symmetries in the latent state space of the generative model learnt under deep active inference. In particular, we focus on object-centric representations, which are trained from pixels to predict novel object views as the agent moves its viewpoint.
  arXiv  Detail & Related papers  (2023-04-14T10:21:26Z)
• Towards fully covariant machine learning [0.0]
  In machine learning, the most visible passive symmetry is the relabeling or permutation symmetry of graphs. We discuss dos and don'ts for machine learning practice if passive symmetries are to be respected.
  arXiv  Detail & Related papers  (2023-01-31T16:01:12Z)
• On Binding Objects to Symbols: Learning Physical Concepts to Understand Real from Fake [155.6741526791004]
  We revisit the classic signal-to-symbol barrier in light of the remarkable ability of deep neural networks to generate synthetic data. We characterize physical objects as abstract concepts and use the previous analysis to show that physical objects can be encoded by finite architectures. We conclude that binding physical entities to digital identities is possible in finite time with finite resources.
  arXiv  Detail & Related papers  (2022-07-25T17:21:59Z)
• An Enactivist-Inspired Mathematical Model of Cognition [5.8010446129208155]
  We formulate five basic tenets of enactivist cognitive science that we have carefully identified in the relevant literature. We then develop a mathematical framework to talk about cognitive systems which complies with these enactivist tenets.
  arXiv  Detail & Related papers  (2022-06-10T13:03:47Z)
• Symmetry Group Equivariant Architectures for Physics [52.784926970374556]
  In the domain of machine learning, an awareness of symmetries has driven impressive performance breakthroughs. We argue that both the physics community and the broader machine learning community have much to understand.
  arXiv  Detail & Related papers  (2022-03-11T18:27:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.