Compositional learning of functions in humans and machines

• URL: http://arxiv.org/abs/2403.12201v1
• Date: Mon, 18 Mar 2024 19:22:53 GMT
• Title: Compositional learning of functions in humans and machines
• Authors: Yanli Zhou, Brenden M. Lake, Adina Williams,
• Abstract summary: We develop a function learning paradigm to explore the capacity of humans and neural network models in learning and reasoning with compositional functions. Our findings indicate that humans can make zero-shot generalizations on novel visual function compositions across interaction conditions. A comparison with a neural network model on the same task reveals that, through the meta-learning for compositionality (MLC) approach, a standard sequence-to-sequence Transformer can mimic human generalization patterns in composing functions.
• Score: 23.583544271543033
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The ability to learn and compose functions is foundational to efficient learning and reasoning in humans, enabling flexible generalizations such as creating new dishes from known cooking processes. Beyond sequential chaining of functions, existing linguistics literature indicates that humans can grasp more complex compositions with interacting functions, where output production depends on context changes induced by different function orderings. Extending the investigation into the visual domain, we developed a function learning paradigm to explore the capacity of humans and neural network models in learning and reasoning with compositional functions under varied interaction conditions. Following brief training on individual functions, human participants were assessed on composing two learned functions, in ways covering four main interaction types, including instances in which the application of the first function creates or removes the context for applying the second function. Our findings indicate that humans can make zero-shot generalizations on novel visual function compositions across interaction conditions, demonstrating sensitivity to contextual changes. A comparison with a neural network model on the same task reveals that, through the meta-learning for compositionality (MLC) approach, a standard sequence-to-sequence Transformer can mimic human generalization patterns in composing functions.

Related papers

• Adaptive Language-Guided Abstraction from Contrastive Explanations [53.48583372522492]
  It is necessary to determine which features of the environment are relevant before determining how these features should be used to compute reward. End-to-end methods for joint feature and reward learning often yield brittle reward functions that are sensitive to spurious state features. This paper describes a method named ALGAE which alternates between using language models to iteratively identify human-meaningful features.
  arXiv  Detail & Related papers  (2024-09-12T16:51:58Z)
• A Pattern Language for Machine Learning Tasks [0.0]
  We view objective functions as constraints on the behaviour of learners. We develop a formal graphical language that allows us to separate the core tasks of a behaviour from its implementation details. As proof-of-concept, we design a novel task that enables converting classifiers into generative models we call "manipulators"
  arXiv  Detail & Related papers  (2024-07-02T16:50:27Z)
• Cognitive Evolutionary Learning to Select Feature Interactions for Recommender Systems [59.117526206317116]
  We show that CELL can adaptively evolve into different models for different tasks and data. Experiments on four real-world datasets demonstrate that CELL significantly outperforms state-of-the-art baselines.
  arXiv  Detail & Related papers  (2024-05-29T02:35:23Z)
• Emergent Modularity in Pre-trained Transformers [127.08792763817496]
  We consider two main characteristics of modularity: functional specialization of neurons and function-based neuron grouping. We study how modularity emerges during pre-training, and find that the modular structure is stabilized at the early stage. It suggests that Transformers first construct the modular structure and then learn fine-grained neuron functions.
  arXiv  Detail & Related papers  (2023-05-28T11:02:32Z)
• Unification of popular artificial neural network activation functions [0.0]
  We present a unified representation of the most popular neural network activation functions. Adopting Mittag-Leffler functions of fractional calculus, we propose a flexible and compact functional form.
  arXiv  Detail & Related papers  (2023-02-21T21:20:59Z)
• Feature Extraction Functions for Neural Logic Rule Learning [4.181432858358386]
  We propose functions for integrating human knowledge abstracted as logic rules into the predictive behavior of a neural network. Unlike other existing neural logic approaches, the programmatic nature of these functions implies that they do not require any kind of special mathematical encoding.
  arXiv  Detail & Related papers  (2020-08-14T12:35:07Z)
• UNIPoint: Universally Approximating Point Processes Intensities [125.08205865536577]
  We provide a proof that a class of learnable functions can universally approximate any valid intensity function. We implement UNIPoint, a novel neural point process model, using recurrent neural networks to parameterise sums of basis function upon each event.
  arXiv  Detail & Related papers  (2020-07-28T09:31:56Z)
• Compositional Explanations of Neurons [52.71742655312625]
  We describe a procedure for explaining neurons in deep representations by identifying compositional logical concepts. We use this procedure to answer several questions on interpretability in models for vision and natural language processing.
  arXiv  Detail & Related papers  (2020-06-24T20:37:05Z)
• Compositional Generalization by Learning Analytical Expressions [87.15737632096378]
  A memory-augmented neural model is connected with analytical expressions to achieve compositional generalization. Experiments on the well-known benchmark SCAN demonstrate that our model seizes a great ability of compositional generalization.
  arXiv  Detail & Related papers  (2020-06-18T15:50:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.