Navigating High Dimensional Concept Space with Metalearning

• URL: http://arxiv.org/abs/2508.01948v1
• Date: Mon, 28 Jul 2025 18:01:38 GMT
• Title: Navigating High Dimensional Concept Space with Metalearning
• Authors: Max Gupta,
• Abstract summary: This work investigates whether gradient-based meta-learning can equip neural networks with inductive biases for efficient few-shot acquisition of discrete concepts.<n>We show that meta-learners are much better able to handle compositional complexity than featural complexity.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Rapidly learning abstract concepts from limited examples is a hallmark of human intelligence. This work investigates whether gradient-based meta-learning can equip neural networks with inductive biases for efficient few-shot acquisition of discrete concepts. We compare meta-learning methods against a supervised learning baseline on Boolean tasks generated by a probabilistic context-free grammar (PCFG). By systematically varying concept dimensionality (number of features) and compositionality (depth of grammar recursion), we identify regimes in which meta-learning robustly improves few-shot concept learning. We find improved performance and sample efficiency by training a multilayer perceptron (MLP) across concept spaces increasing in dimensional and compositional complexity. We are able to show that meta-learners are much better able to handle compositional complexity than featural complexity and establish an empirical analysis demonstrating how featural complexity shapes 'concept basins' of the loss landscape, allowing curvature-aware optimization to be more effective than first order methods. We see that we can robustly increase generalization on complex concepts by increasing the number of adaptation steps in meta-SGD, encouraging exploration of rougher loss basins. Overall, this work highlights the intricacies of learning compositional versus featural complexity in high dimensional concept spaces and provides a road to understanding the role of 2nd order methods and extended gradient adaptation in few-shot concept learning.

Related papers

• Human-like compositional learning of visually-grounded concepts using synthetic environments [6.461018127662044]
  We investigate how humans learn to compose concept classes and ground visual cues through trial and error.<n>We design a 3D synthetic environment in which an agent learns, via reinforcement, to navigate to a target specified by a natural language instruction.<n>We show that reinforcement learning agents can ground determiner concepts to visual targets but struggle with more complex prepositional concepts.
  arXiv  Detail & Related papers  (2025-04-09T06:33:28Z)
• Enabling Systematic Generalization in Abstract Spatial Reasoning through Meta-Learning for Compositionality [20.958479821810762]
  We extend the approach of meta-learning for compositionality to the domain of abstract spatial reasoning.<n>Our results show that a transformer-based encoder-decoder model, trained via meta-learning for compositionality, can systematically generalize to previously unseen transformation compositions.
  arXiv  Detail & Related papers  (2025-04-02T07:56:39Z)
• Deep Learning Through A Telescoping Lens: A Simple Model Provides Empirical Insights On Grokking, Gradient Boosting & Beyond [61.18736646013446]
  In pursuit of a deeper understanding of its surprising behaviors, we investigate the utility of a simple yet accurate model of a trained neural network. Across three case studies, we illustrate how it can be applied to derive new empirical insights on a diverse range of prominent phenomena.
  arXiv  Detail & Related papers  (2024-10-31T22:54:34Z)
• Coding for Intelligence from the Perspective of Category [66.14012258680992]
  Coding targets compressing and reconstructing data, and intelligence. Recent trends demonstrate the potential homogeneity of these two fields. We propose a novel problem of Coding for Intelligence from the category theory view.
  arXiv  Detail & Related papers  (2024-07-01T07:05:44Z)
• Sketch-Plan-Generalize: Learning and Planning with Neuro-Symbolic Programmatic Representations for Inductive Spatial Concepts [6.708785987015999]
  We develop an approach to learn personalized concepts from a limited number of demonstrations.<n>Our pipeline facilitates generalization and modular re-use, enabling continual concept learning.
  arXiv  Detail & Related papers  (2024-04-11T14:09:41Z)
• Concept Discovery for Fast Adapatation [42.81705659613234]
  We introduce concept discovery to the few-shot learning problem, where we achieve more effective adaptation by meta-learning the structure among the data features. Our proposed method Concept-Based Model-Agnostic Meta-Learning (COMAML) has been shown to achieve consistent improvements in the structured data for both synthesized datasets and real-world datasets.
  arXiv  Detail & Related papers  (2023-01-19T02:33:58Z)
• Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyond [114.39616146985001]
  In machine learning and computer vision fields, despite the different motivations and mechanisms, a lot of complex problems contain a series of closely related subproblms. In this paper, we first uniformly express these complex learning and vision problems from the perspective of Bi-Level Optimization (BLO) Then we construct a value-function-based single-level reformulation and establish a unified algorithmic framework to understand and formulate mainstream gradient-based BLO methodologies.
  arXiv  Detail & Related papers  (2021-01-27T16:20:23Z)
• Importance Weighted Policy Learning and Adaptation [89.46467771037054]
  We study a complementary approach which is conceptually simple, general, modular and built on top of recent improvements in off-policy learning. The framework is inspired by ideas from the probabilistic inference literature and combines robust off-policy learning with a behavior prior. Our approach achieves competitive adaptation performance on hold-out tasks compared to meta reinforcement learning baselines and can scale to complex sparse-reward scenarios.
  arXiv  Detail & Related papers  (2020-09-10T14:16:58Z)
• Concept Learners for Few-Shot Learning [76.08585517480807]
  We propose COMET, a meta-learning method that improves generalization ability by learning to learn along human-interpretable concept dimensions. We evaluate our model on few-shot tasks from diverse domains, including fine-grained image classification, document categorization and cell type annotation.
  arXiv  Detail & Related papers  (2020-07-14T22:04:17Z)
• Provable Representation Learning for Imitation Learning via Bi-level Optimization [60.059520774789654]
  A common strategy in modern learning systems is to learn a representation that is useful for many tasks. We study this strategy in the imitation learning setting for Markov decision processes (MDPs) where multiple experts' trajectories are available. We instantiate this framework for the imitation learning settings of behavior cloning and observation-alone.
  arXiv  Detail & Related papers  (2020-02-24T21:03:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.