Solving ARC visual analogies with neural embeddings and vector
arithmetic: A generalized method
- URL: http://arxiv.org/abs/2311.08083v1
- Date: Tue, 14 Nov 2023 11:10:46 GMT
- Title: Solving ARC visual analogies with neural embeddings and vector
arithmetic: A generalized method
- Authors: Luca H. Thoms, Karel A. Veldkamp, Hannes Rosenbusch and Claire E.
Stevenson
- Abstract summary: Analogical reasoning derives information from known relations and generalizes this information to similar yet unfamiliar situations.
One of the first generalized ways in which deep learning models were able to solve verbal analogies was through vector arithmetic of word embeddings.
This project focuses on visual analogical reasoning and applies the initial generalized mechanism used to solve verbal analogies to the visual realm.
- Score: 0.0
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Analogical reasoning derives information from known relations and generalizes
this information to similar yet unfamiliar situations. One of the first
generalized ways in which deep learning models were able to solve verbal
analogies was through vector arithmetic of word embeddings, essentially
relating words that were mapped to a vector space (e.g., king - man + woman =
__?). In comparison, most attempts to solve visual analogies are still
predominantly task-specific and less generalizable. This project focuses on
visual analogical reasoning and applies the initial generalized mechanism used
to solve verbal analogies to the visual realm. Taking the Abstraction and
Reasoning Corpus (ARC) as an example to investigate visual analogy solving, we
use a variational autoencoder (VAE) to transform ARC items into low-dimensional
latent vectors, analogous to the word embeddings used in the verbal approaches.
Through simple vector arithmetic, underlying rules of ARC items are discovered
and used to solve them. Results indicate that the approach works well on simple
items with fewer dimensions (i.e., few colors used, uniform shapes), similar
input-to-output examples, and high reconstruction accuracy on the VAE.
Predictions on more complex items showed stronger deviations from expected
outputs, although, predictions still often approximated parts of the item's
rule set. Error patterns indicated that the model works as intended. On the
official ARC paradigm, the model achieved a score of 2% (cf. current world
record is 21%) and on ConceptARC it scored 8.8%. Although the methodology
proposed involves basic dimensionality reduction techniques and standard vector
arithmetic, this approach demonstrates promising outcomes on ARC and can easily
be generalized to other abstract visual reasoning tasks.
Related papers
- Tackling the Abstraction and Reasoning Corpus (ARC) with Object-centric
Models and the MDL Principle [0.0]
We introduce object-centric models that are in line with the natural programs produced by humans.
Our models can not only perform predictions, but also provide joint descriptions for input/output pairs.
A diverse range of tasks are solved, and the learned models are similar to the natural programs.
arXiv Detail & Related papers (2023-11-01T14:25:51Z) - LLMs and the Abstraction and Reasoning Corpus: Successes, Failures, and
the Importance of Object-based Representations [50.431003245201644]
We show that GPT-4 is unable to "reason" perfectly within non-language domains such as the 1D-ARC or a simple ARC subset.
We propose an object-based representation that is obtained through an external tool, resulting in nearly doubling the performance on solved ARC tasks and near-perfect scores on the easier 1D-ARC.
arXiv Detail & Related papers (2023-05-26T16:32:17Z) - Interpretability at Scale: Identifying Causal Mechanisms in Alpaca [62.65877150123775]
We use Boundless DAS to efficiently search for interpretable causal structure in large language models while they follow instructions.
Our findings mark a first step toward faithfully understanding the inner-workings of our ever-growing and most widely deployed language models.
arXiv Detail & Related papers (2023-05-15T17:15:40Z) - How do Variational Autoencoders Learn? Insights from Representational
Similarity [2.969705152497174]
We study the internal behaviour of Variational Autoencoders (VAEs) using representational similarity techniques.
Using the CKA and Procrustes similarities, we found that the encoders' representations are learned long before the decoders'.
arXiv Detail & Related papers (2022-05-17T14:31:57Z) - Visual Abductive Reasoning [85.17040703205608]
Abductive reasoning seeks the likeliest possible explanation for partial observations.
We propose a new task and dataset, Visual Abductive Reasoning ( VAR), for examining abductive reasoning ability of machine intelligence in everyday visual situations.
arXiv Detail & Related papers (2022-03-26T10:17:03Z) - Learning Algebraic Representation for Systematic Generalization in
Abstract Reasoning [109.21780441933164]
We propose a hybrid approach to improve systematic generalization in reasoning.
We showcase a prototype with algebraic representation for the abstract spatial-temporal task of Raven's Progressive Matrices (RPM)
We show that the algebraic representation learned can be decoded by isomorphism to generate an answer.
arXiv Detail & Related papers (2021-11-25T09:56:30Z) - DiGS : Divergence guided shape implicit neural representation for
unoriented point clouds [36.60407995156801]
Shape implicit neural representations (INRs) have recently shown to be effective in shape analysis and reconstruction tasks.
We propose a divergence guided shape representation learning approach that does not require normal vectors as input.
arXiv Detail & Related papers (2021-06-21T02:10:03Z) - Revisiting Contrastive Methods for Unsupervised Learning of Visual
Representations [78.12377360145078]
Contrastive self-supervised learning has outperformed supervised pretraining on many downstream tasks like segmentation and object detection.
In this paper, we first study how biases in the dataset affect existing methods.
We show that current contrastive approaches work surprisingly well across: (i) object- versus scene-centric, (ii) uniform versus long-tailed and (iii) general versus domain-specific datasets.
arXiv Detail & Related papers (2021-06-10T17:59:13Z) - Improving Aspect-based Sentiment Analysis with Gated Graph Convolutional
Networks and Syntax-based Regulation [89.38054401427173]
Aspect-based Sentiment Analysis (ABSA) seeks to predict the sentiment polarity of a sentence toward a specific aspect.
dependency trees can be integrated into deep learning models to produce the state-of-the-art performance for ABSA.
We propose a novel graph-based deep learning model to overcome these two issues.
arXiv Detail & Related papers (2020-10-26T07:36:24Z) - Analyzing Knowledge Graph Embedding Methods from a Multi-Embedding
Interaction Perspective [3.718476964451589]
Real-world knowledge graphs are usually incomplete, so knowledge graph embedding methods have been proposed to address this issue.
These methods represent entities and relations as embedding vectors in semantic space and predict the links between them.
We propose a new multi-embedding model based on quaternion algebra and show that it achieves promising results using popular benchmarks.
arXiv Detail & Related papers (2019-03-27T13:09:16Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.