Learning Algebraic Representation for Systematic Generalization in Abstract Reasoning

• URL: http://arxiv.org/abs/2111.12990v1
• Date: Thu, 25 Nov 2021 09:56:30 GMT
• Title: Learning Algebraic Representation for Systematic Generalization in Abstract Reasoning
• Authors: Chi Zhang, Sirui Xie, Baoxiong Jia, Ying Nian Wu, Song-Chun Zhu, Yixin Zhu
• Abstract summary: We propose a hybrid approach to improve systematic generalization in reasoning. We showcase a prototype with algebraic representation for the abstract spatial-temporal task of Raven's Progressive Matrices (RPM) We show that the algebraic representation learned can be decoded by isomorphism to generate an answer.
• Score: 109.21780441933164
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Is intelligence realized by connectionist or classicist? While connectionist approaches have achieved superhuman performance, there has been growing evidence that such task-specific superiority is particularly fragile in systematic generalization. This observation lies in the central debate between connectionist and classicist, wherein the latter continually advocates an algebraic treatment in cognitive architectures. In this work, we follow the classicist's call and propose a hybrid approach to improve systematic generalization in reasoning. Specifically, we showcase a prototype with algebraic representation for the abstract spatial-temporal reasoning task of Raven's Progressive Matrices (RPM) and present the ALgebra-Aware Neuro-Semi-Symbolic (ALANS) learner. The ALANS learner is motivated by abstract algebra and the representation theory. It consists of a neural visual perception frontend and an algebraic abstract reasoning backend: the frontend summarizes the visual information from object-based representation, while the backend transforms it into an algebraic structure and induces the hidden operator on the fly. The induced operator is later executed to predict the answer's representation, and the choice most similar to the prediction is selected as the solution. Extensive experiments show that by incorporating an algebraic treatment, the ALANS learner outperforms various pure connectionist models in domains requiring systematic generalization. We further show that the algebraic representation learned can be decoded by isomorphism to generate an answer.

Related papers

• VisualPredicator: Learning Abstract World Models with Neuro-Symbolic Predicates for Robot Planning [86.59849798539312]
  We present Neuro-Symbolic Predicates, a first-order abstraction language that combines the strengths of symbolic and neural knowledge representations. We show that our approach offers better sample complexity, stronger out-of-distribution generalization, and improved interpretability.
  arXiv  Detail & Related papers  (2024-10-30T16:11:05Z)
• A Cognitively-Inspired Neural Architecture for Visual Abstract Reasoning Using Contrastive Perceptual and Conceptual Processing [14.201935774784632]
  We introduce a new neural architecture for solving visual abstract reasoning tasks inspired by human cognition. Inspired by this principle, our architecture models visual abstract reasoning as an iterative, self-contrasting learning process. Experiments on the machine learning dataset RAVEN show that CPCNet achieves higher accuracy than all previously published models.
  arXiv  Detail & Related papers  (2023-09-19T11:18:01Z)
• Vector-based Representation is the Key: A Study on Disentanglement and Compositional Generalization [77.57425909520167]
  We show that it is possible to achieve both good concept recognition and novel concept composition. We propose a method to reform the scalar-based disentanglement works to be vector-based to increase both capabilities.
  arXiv  Detail & Related papers  (2023-05-29T13:05:15Z)
• On the Complexity of Representation Learning in Contextual Linear Bandits [110.84649234726442]
  We show that representation learning is fundamentally more complex than linear bandits. In particular, learning with a given set of representations is never simpler than learning with the worst realizable representation in the set.
  arXiv  Detail & Related papers  (2022-12-19T13:08:58Z)
• Knowledgebra: An Algebraic Learning Framework for Knowledge Graph [15.235089177507897]
  Knowledge graph (KG) representation learning aims to encode entities and relations into dense continuous vector spaces such that knowledge contained in a dataset could be consistently represented. We developed a mathematical language for KG based on an observation of their inherent algebraic structure, which we termed as Knowledgebra. We implemented an instantiation model, SemE, using simple matrix semigroups, which exhibits state-of-the-art performance on standard datasets.
  arXiv  Detail & Related papers  (2022-04-15T04:53:47Z)
• Learning Algebraic Recombination for Compositional Generalization [71.78771157219428]
  We propose LeAR, an end-to-end neural model to learn algebraic recombination for compositional generalization. Key insight is to model the semantic parsing task as a homomorphism between a latent syntactic algebra and a semantic algebra. Experiments on two realistic and comprehensive compositional generalization demonstrate the effectiveness of our model.
  arXiv  Detail & Related papers  (2021-07-14T07:23:46Z)
• Abstract Spatial-Temporal Reasoning via Probabilistic Abduction and Execution [97.50813120600026]
  Spatial-temporal reasoning is a challenging task in Artificial Intelligence (AI) Recent works have focused on an abstract reasoning task of this kind -- Raven's Progressive Matrices ( RPM) We propose a neuro-symbolic Probabilistic Abduction and Execution learner (PrAE) learner.
  arXiv  Detail & Related papers  (2021-03-26T02:42:18Z)
• Knowledge Hypergraph Embedding Meets Relational Algebra [13.945694569456665]
  We propose a simple embedding-based model called ReAlE that performs link prediction in knowledge hypergraphs. We show theoretically that ReAlE is fully expressive and provide proofs and empirical evidence that it can represent a large subset of the primitive relational algebra operations.
  arXiv  Detail & Related papers  (2021-02-18T18:57:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.