Manifold Bandits: Bayesian Curriculum Learning over the Latent Geometry of Large Language Models

• URL: http://arxiv.org/abs/2606.19750v1
• Date: Thu, 18 Jun 2026 03:31:19 GMT
• Title: Manifold Bandits: Bayesian Curriculum Learning over the Latent Geometry of Large Language Models
• Abstract summary: Reinforcement learning (RL) is a central approach for improving reasoning capabilities in large language models.<n>We frame problem sampling as a manifold-structured bandit problem with endogenous non-stationarity.<n>We find that different sampling strategies induce non-trivial tradeoffs between productivity (learning signal), diversity (coverage of the task manifold), and utility.
• Score: 11.051532211938556
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reinforcement learning (RL) is a central approach for improving reasoning capabilities in large language models (LLMs), where training efficiency depends critically on how problems are sampled during optimization. Existing adaptive curriculum learning methods typically prioritize prompts of intermediate difficulty, treating problem selection as a standard bandit problem with independent arms and overlooking the structured, heterogeneous nature of the task space. In this work, we frame problem sampling as a manifold-structured bandit problem with endogenous non-stationarity: problems are related through the model's latent representation space, and sampling decisions can steer how learning signals evolve across that space. To operationalize this perspective, we introduce Bayesian Manifold Curriculum (BMC), a structure-aware framework that organizes problems into a hierarchical task tree and applies Bayesian learning to guide sampling. Empirically, we find that different sampling strategies induce non-trivial tradeoffs between productivity (learning signal), diversity (coverage of the task manifold), and utility (evaluation relevance). These results show that prioritizing difficulty alone is insufficient for strong downstream performance, highlighting the importance of incorporating structure and type-awareness into problem sampling.

Related papers

• UniICL: Systematizing Unified Multimodal In-context Learning through a Capability-Oriented Taxonomy [99.95663439922937]
  In-context Learning efficacy is often non-monotonic and highly task-dependent.<n>We introduce a six-level capability-oriented taxonomy that categorizes the functional role of demonstrations.<n>We construct UniICL-760K, a large-scale corpus featuring curated 8-shot In-context Learning episodes.<n>As an architectural intervention, we propose the Context-Adaptive Prototype Modulator.
  arXiv  Detail & Related papers  (2026-03-25T18:09:33Z)
• MSA-Thinker: Discrimination-Calibration Reasoning with Hint-Guided Reinforcement Learning for Multimodal Sentiment Analysis [5.1150258716324055]
  Multimodal sentiment analysis aims to understand human emotions by integrating textual, auditory, and visual modalities.<n>Existing methods incorporating Chain-of-Thought (CoT) reasoning are hindered by high annotation costs.<n>We propose a novel training framework that integrates structured Discrimination-Calibration (DC) reasoning with Hint-based Reinforcement Learning.
  arXiv  Detail & Related papers  (2026-03-10T12:48:41Z)
• Bidirectional Curriculum Generation: A Multi-Agent Framework for Data-Efficient Mathematical Reasoning [16.95900718416944]
  We introduce a novel Bidirectional Curriculum Generation framework to maximize the instructional value of every training sample.<n>Unlike rigid trajectories, our multi-agent ecosystem mimics adaptive pedagogy to establish a closed feedback loop.<n>This mechanism ensures that the model consumes only the most effective data at any given stage.
  arXiv  Detail & Related papers  (2026-03-05T12:49:21Z)
• Provable Benefit of Curriculum in Transformer Tree-Reasoning Post-Training [76.12556589212666]
  We show that curriculum post-training avoids the exponential complexity bottleneck.<n>Under outcome-only reward signals, reinforcement learning finetuning achieves high accuracy with sample complexity.<n>We establish guarantees for test-time scaling, where curriculum-aware querying reduces both reward oracle calls and sampling cost from exponential to order.
  arXiv  Detail & Related papers  (2025-11-10T18:29:54Z)
• In-Context Learning Is Provably Bayesian Inference: A Generalization Theory for Meta-Learning [51.56484100374058]
  We introduce a principled risk decomposition that separates the total ICL risk into two components: Bayes Gap and Posterior Variance.<n>For a uniform-attention Transformer, we derive a non-asymptotic upper bound on this gap, which explicitly clarifies the dependence on the number of pretraining prompts.<n>The Posterior Variance is a model-independent risk representing the intrinsic task uncertainty.
  arXiv  Detail & Related papers  (2025-10-13T03:42:31Z)
• Tailored Teaching with Balanced Difficulty: Elevating Reasoning in Multimodal Chain-of-Thought via Prompt Curriculum [39.57901536686932]
  Multimodal Chain-of-Thought (MCoT) prompting is often limited by the use of randomly or manually selected examples.<n>We propose a novel framework inspired by the pedagogical principle of "tailored teaching with balanced difficulty"<n>Our approach integrates two complementary signals: model-perceived difficulty, quantified through prediction disagreement in an active learning setup, and intrinsic sample complexity, which measures the inherent difficulty of each question-image pair independently of any model.
  arXiv  Detail & Related papers  (2025-08-26T04:32:15Z)
• On Distributional Dependent Performance of Classical and Neural Routing Solvers [5.359176539960004]
  NCO aims to learn to solve a class of problems by learning the underlying distribution of problem instances.<n>This work explores a novel approach to formulate the distribution of problem instances to learn from and, more importantly, plant a structure in the sampled problem instances.<n>We evaluate representative NCO methods and specialized Operation Research metas on this novel task and demonstrate that the performance gap between neural routing solvers and highly specialized meta-heuristics decreases when learning from sub-samples drawn from a fixed base node distribution.
  arXiv  Detail & Related papers  (2025-08-04T15:17:08Z)
• Feature-Based vs. GAN-Based Learning from Demonstrations: When and Why [50.191655141020505]
  This survey provides a comparative analysis of feature-based and GAN-based approaches to learning from demonstrations.<n>We argue that the dichotomy between feature-based and GAN-based methods is increasingly nuanced.
  arXiv  Detail & Related papers  (2025-07-08T11:45:51Z)
• Unlocking the Potential of Difficulty Prior in RL-based Multimodal Reasoning [69.64809103333839]
  We investigate how explicitly modeling problem's difficulty prior information shapes the effectiveness of reinforcement learning based fine-tuning for multimodal reasoning.<n>Our approach demonstrates significant performances across various multi-modal mathematical reasoning benchmarks with only 2K+0.6K two-stage training data.
  arXiv  Detail & Related papers  (2025-05-19T15:43:10Z)
• Causally Aligned Curriculum Learning [69.11672390876763]
  This paper studies the problem of curriculum RL through causal lenses.<n>We derive a sufficient graphical condition characterizing causally aligned source tasks.<n>We develop an efficient algorithm to generate a causally aligned curriculum.
  arXiv  Detail & Related papers  (2025-03-21T02:20:38Z)
• Learning Robust Representation for Clustering through Locality Preserving Variational Discriminative Network [16.259673823482665]
  Variational Deep Embedding achieves great success in various clustering tasks. VaDE suffers from two problems: 1) it is fragile to the input noise; 2) it ignores the locality information between the neighboring data points. We propose a joint learning framework that improves VaDE with a robust embedding discriminator and a local structure constraint.
  arXiv  Detail & Related papers  (2020-12-25T02:31:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.