Compositional diversity in visual concept learning

• URL: http://arxiv.org/abs/2305.19374v1
• Date: Tue, 30 May 2023 19:30:50 GMT
• Title: Compositional diversity in visual concept learning
• Authors: Yanli Zhou, Reuben Feinman, Brenden M. Lake
• Abstract summary: Humans leverage compositionality to efficiently learn new concepts, understanding how familiar parts can combine together to form novel objects. Here, we study how people classify and generate alien figures'' with rich relational structure. We develop a Bayesian program induction model which searches for the best programs for generating the candidate visual figures.
• Score: 18.907108368038216
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Humans leverage compositionality to efficiently learn new concepts, understanding how familiar parts can combine together to form novel objects. In contrast, popular computer vision models struggle to make the same types of inferences, requiring more data and generalizing less flexibly than people do. Here, we study these distinctively human abilities across a range of different types of visual composition, examining how people classify and generate ``alien figures'' with rich relational structure. We also develop a Bayesian program induction model which searches for the best programs for generating the candidate visual figures, utilizing a large program space containing different compositional mechanisms and abstractions. In few shot classification tasks, we find that people and the program induction model can make a range of meaningful compositional generalizations, with the model providing a strong account of the experimental data as well as interpretable parameters that reveal human assumptions about the factors invariant to category membership (here, to rotation and changing part attachment). In few shot generation tasks, both people and the models are able to construct compelling novel examples, with people behaving in additional structured ways beyond the model capabilities, e.g. making choices that complete a set or reconfiguring existing parts in highly novel ways. To capture these additional behavioral patterns, we develop an alternative model based on neuro-symbolic program induction: this model also composes new concepts from existing parts yet, distinctively, it utilizes neural network modules to successfully capture residual statistical structure. Together, our behavioral and computational findings show how people and models can produce a rich variety of compositional behavior when classifying and generating visual objects.

Related papers

• Automatic Discovery of Visual Circuits [66.99553804855931]
  We explore scalable methods for extracting the subgraph of a vision model's computational graph that underlies recognition of a specific visual concept. We find that our approach extracts circuits that causally affect model output, and that editing these circuits can defend large pretrained models from adversarial attacks.
  arXiv  Detail & Related papers  (2024-04-22T17:00:57Z)
• Corpus Considerations for Annotator Modeling and Scaling [9.263562546969695]
  We show that the commonly used user token model consistently outperforms more complex models. Our findings shed light on the relationship between corpus statistics and annotator modeling performance.
  arXiv  Detail & Related papers  (2024-04-02T22:27:24Z)
• A Recursive Bateson-Inspired Model for the Generation of Semantic Formal Concepts from Spatial Sensory Data [77.34726150561087]
  This paper presents a new symbolic-only method for the generation of hierarchical concept structures from complex sensory data. The approach is based on Bateson's notion of difference as the key to the genesis of an idea or a concept. The model is able to produce fairly rich yet human-readable conceptual representations without training.
  arXiv  Detail & Related papers  (2023-07-16T15:59:13Z)
• Language Model Cascades [72.18809575261498]
  Repeated interactions at test-time with a single model, or the composition of multiple models together, further expands capabilities. Cases with control flow and dynamic structure require techniques from probabilistic programming. We formalize several existing techniques from this perspective, including scratchpads / chain of thought, verifiers, STaR, selection-inference, and tool use.
  arXiv  Detail & Related papers  (2022-07-21T07:35:18Z)
• Model LEGO: Creating Models Like Disassembling and Assembling Building Blocks [53.09649785009528]
  In this paper, we explore a paradigm that does not require training to obtain new models. Similar to the birth of CNN inspired by receptive fields in the biological visual system, we propose Model Disassembling and Assembling. For model assembling, we present the alignment padding strategy and parameter scaling strategy to construct a new model tailored for a specific task.
  arXiv  Detail & Related papers  (2022-03-25T05:27:28Z)
• Lensing Machines: Representing Perspective in Latent Variable Models [1.0878040851638]
  We introduce lensing, a mixed initiative technique to extract lenses or mappings between machine learned representations and perspectives of human experts. We apply lensing for two classes of latent variable models: a mixed membership model and a matrix factorization model in the context of two mental health applications.
  arXiv  Detail & Related papers  (2022-01-20T21:42:20Z)
• Meta-brain Models: biologically-inspired cognitive agents [0.0]
  We propose a computational approach we call meta-brain models. We will propose combinations of layers composed using specialized types of models. We will conclude by proposing next steps in the development of this flexible and open-source approach.
  arXiv  Detail & Related papers  (2021-08-31T05:20:53Z)
• Model-agnostic multi-objective approach for the evolutionary discovery of mathematical models [55.41644538483948]
  In modern data science, it is more interesting to understand the properties of the model, which parts could be replaced to obtain better results. We use multi-objective evolutionary optimization for composite data-driven model learning to obtain the algorithm's desired properties.
  arXiv  Detail & Related papers  (2021-07-07T11:17:09Z)
• Flexible Compositional Learning of Structured Visual Concepts [17.665938343060112]
  We study how people learn different types of visual compositions, using abstract visual forms with rich relational structure. We find that people can make meaningful compositional generalizations from just a few examples in a variety of scenarios. Unlike past work examining special cases of compositionality, our work shows how a single computational approach can account for many distinct types of compositional generalization.
  arXiv  Detail & Related papers  (2021-05-20T15:48:05Z)
• Learning abstract structure for drawing by efficient motor program induction [52.13961975752941]
  We develop a naturalistic drawing task to study how humans rapidly acquire structured prior knowledge. We show that people spontaneously learn abstract drawing procedures that support generalization. We propose a model of how learners can discover these reusable drawing programs.
  arXiv  Detail & Related papers  (2020-08-08T13:31:14Z)
• Learning Task-General Representations with Generative Neuro-Symbolic Modeling [22.336243882030026]
  We develop a generative neuro-symbolic (GNS) model of handwritten character concepts. The correlations between parts are modeled with neural network subroutines, allowing the model to learn directly from raw data. In a subsequent evaluation, our GNS model uses probabilistic inference to learn rich conceptual representations from a single training image.
  arXiv  Detail & Related papers  (2020-06-25T14:41:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.