MetaScale: Test-Time Scaling with Evolving Meta-Thoughts

• URL: http://arxiv.org/abs/2503.13447v1
• Date: Mon, 17 Mar 2025 17:59:54 GMT
• Title: MetaScale: Test-Time Scaling with Evolving Meta-Thoughts
• Authors: Qin Liu, Wenxuan Zhou, Nan Xu, James Y. Huang, Fei Wang, Sheng Zhang, Hoifung Poon, Muhao Chen,
• Abstract summary: Experimental results demonstrate that MetaScale consistently outperforms standard inference approaches.<n> METASCALE scales more effectively with increasing sampling budgets and produces more structured, expert-level responses.
• Score: 51.35594569020857
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: One critical challenge for large language models (LLMs) for making complex reasoning is their reliance on matching reasoning patterns from training data, instead of proactively selecting the most appropriate cognitive strategy to solve a given task. Existing approaches impose fixed cognitive structures that enhance performance in specific tasks but lack adaptability across diverse scenarios. To address this limitation, we introduce METASCALE, a test-time scaling framework based on meta-thoughts -- adaptive thinking strategies tailored to each task. METASCALE initializes a pool of candidate meta-thoughts, then iteratively selects and evaluates them using a multi-armed bandit algorithm with upper confidence bound selection, guided by a reward model. To further enhance adaptability, a genetic algorithm evolves high-reward meta-thoughts, refining and extending the strategy pool over time. By dynamically proposing and optimizing meta-thoughts at inference time, METASCALE improves both accuracy and generalization across a wide range of tasks. Experimental results demonstrate that MetaScale consistently outperforms standard inference approaches, achieving an 11% performance gain in win rate on Arena-Hard for GPT-4o, surpassing o1-mini by 0.9% under style control. Notably, METASCALE scales more effectively with increasing sampling budgets and produces more structured, expert-level responses.

Related papers

• AdaptMMBench: Benchmarking Adaptive Multimodal Reasoning for Mode Selection and Reasoning Process [35.95284812390557]
  We propose AdaptMMBench, a benchmark for adaptive multimodal reasoning across five domains: real-world, OCR, GUI, knowledge, and math.<n>Our evaluation reveals that while adaptive mode selection scales with model capacity, it notably decouples from final accuracy.
  arXiv  Detail & Related papers  (2026-02-02T19:00:27Z)
• Beyond Fast and Slow: Cognitive-Inspired Elastic Reasoning for Large Language Models [39.03483371038282]
  CogER is a framework inspired by human hierarchical reasoning.<n>For queries requiring external tools, we introduce Cognitive Tool-Assisted Reasoning.<n>CogER outperforms state-of-the-art Test-Time scaling methods.
  arXiv  Detail & Related papers  (2025-12-17T05:11:58Z)
• Plan before Solving: Problem-Aware Strategy Routing for Mathematical Reasoning with LLMs [49.995906301946]
  Existing methods usually leverage a fixed strategy to guide Large Language Models (LLMs) to perform mathematical reasoning.<n>Our analysis reveals that the single strategy cannot adapt to problem-specific requirements and thus overlooks the trade-off between effectiveness and efficiency.<n>We propose Planning and Routing through Instance-Specific Modeling (PRISM), a novel framework that decouples mathematical reasoning into two stages: strategy planning and targeted execution.
  arXiv  Detail & Related papers  (2025-09-29T07:22:41Z)
• Meta-Awareness Enhances Reasoning Models: Self-Alignment Reinforcement Learning [38.67622953293653]
  We design a training pipeline that boosts Meta-Awareness via Self-Alignment (MASA)<n>Unlike existing meta-cognitive reasoning models, our method does not require external training sources.<n>Our strategy yields significant improvements in both accuracy and training efficiency on in-domain tasks.
  arXiv  Detail & Related papers  (2025-09-26T14:05:48Z)
• Staying in the Sweet Spot: Responsive Reasoning Evolution via Capability-Adaptive Hint Scaffolding [59.60915947702282]
  Reinforcement learning with verifiable rewards (RLVR) has achieved remarkable success in enhancing the reasoning capabilities of large language models (LLMs)<n>Existing RLVR methods often suffer from exploration inefficiency due to mismatches between the training data's difficulty and the model's capability.<n>We propose SEELE, a novel supervision-aided RLVR framework that dynamically adjusts problem difficulty to stay within the high-efficiency region.
  arXiv  Detail & Related papers  (2025-09-08T17:36:21Z)
• MEMETRON: Metaheuristic Mechanisms for Test-time Response Optimization of Large Language Models [0.6926105253992517]
  Large language models (LLMs) are increasingly used for both open-ended and structured tasks.<n>We introduce MEMETRON, a task-agnostic framework that formulates LLM decoding as a discrete black-box optimization problem.<n>We evaluate our framework on the critical human preference alignment task and demonstrate that it significantly outperforms standard decoding and reranking methods.
  arXiv  Detail & Related papers  (2025-06-10T09:55:53Z)
• PATS: Process-Level Adaptive Thinking Mode Switching [53.53401063490537]
  Current large-language models (LLMs) typically adopt a fixed reasoning strategy, either simple or complex, for all questions, regardless of their difficulty.<n>This neglect of variation in task and reasoning process complexity leads to an imbalance between performance and efficiency.<n>Existing methods attempt to implement training-free fast-slow thinking system switching to handle problems of varying difficulty, but are limited by coarse-grained solution-level strategy adjustments.<n>We propose a novel reasoning paradigm: Process-Level Adaptive Thinking Mode Switching (PATS), which enables LLMs to dynamically adjust their reasoning strategy based on the difficulty of each step, optimizing the balance between
  arXiv  Detail & Related papers  (2025-05-25T17:58:50Z)
• ReMA: Learning to Meta-think for LLMs with Multi-Agent Reinforcement Learning [53.817538122688944]
  We introduce Reinforced Meta-thinking Agents (ReMA) to elicit meta-thinking behaviors from Reasoning of Large Language Models (LLMs)<n>ReMA decouples the reasoning process into two hierarchical agents: a high-level meta-thinking agent responsible for generating strategic oversight and plans, and a low-level reasoning agent for detailed executions.<n> Empirical results from single-turn experiments demonstrate that ReMA outperforms single-agent RL baselines on complex reasoning tasks.
  arXiv  Detail & Related papers  (2025-03-12T16:05:31Z)
• Clustering-based Meta Bayesian Optimization with Theoretical Guarantee [9.821653903127107]
  We propose a scalable and robust meta-BO method designed to address key challenges in heterogeneous and large-scale meta-tasks. Our approach effectively partitions transferred meta-functions into highly homogeneous clusters, learns the geometry-based surrogate prototype, and adaptively synthesizes meta-priors during the online phase.
  arXiv  Detail & Related papers  (2025-03-08T06:46:28Z)
• MR-Ben: A Meta-Reasoning Benchmark for Evaluating System-2 Thinking in LLMs [55.20845457594977]
  Large language models (LLMs) have shown increasing capability in problem-solving and decision-making. We present a process-based benchmark MR-Ben that demands a meta-reasoning skill. Our meta-reasoning paradigm is especially suited for system-2 slow thinking.
  arXiv  Detail & Related papers  (2024-06-20T03:50:23Z)
• RLEMMO: Evolutionary Multimodal Optimization Assisted By Deep Reinforcement Learning [8.389454219309837]
  multimodal optimization problems (MMOP) requires finding all optimal solutions, which is challenging in limited function evaluations. We propose RLEMMO, a Meta-Black-Box Optimization framework, which maintains a population of solutions and incorporates a reinforcement learning agent. With a novel reward mechanism that encourages both quality and diversity, RLEMMO can be effectively trained using a policy gradient algorithm.
  arXiv  Detail & Related papers  (2024-04-12T05:02:49Z)
• Evolving Pareto-Optimal Actor-Critic Algorithms for Generalizability and Stability [67.8426046908398]
  Generalizability and stability are two key objectives for operating reinforcement learning (RL) agents in the real world. This paper presents MetaPG, an evolutionary method for automated design of actor-critic loss functions.
  arXiv  Detail & Related papers  (2022-04-08T20:46:16Z)
• Meta-learning with an Adaptive Task Scheduler [93.63502984214918]
  Existing meta-learning algorithms randomly sample meta-training tasks with a uniform probability. It is likely that tasks are detrimental with noise or imbalanced given a limited number of meta-training tasks. We propose an adaptive task scheduler (ATS) for the meta-training process.
  arXiv  Detail & Related papers  (2021-10-26T22:16:35Z)
• Meta Learning Black-Box Population-Based Optimizers [0.0]
  We propose the use of meta-learning to infer population-based blackbox generalizations. We show that the meta-loss function encourages a learned algorithm to alter its search behavior so that it can easily fit into a new context.
  arXiv  Detail & Related papers  (2021-03-05T08:13:25Z)
• Meta-Learning with Neural Tangent Kernels [58.06951624702086]
  We propose the first meta-learning paradigm in the Reproducing Kernel Hilbert Space (RKHS) induced by the meta-model's Neural Tangent Kernel (NTK) Within this paradigm, we introduce two meta-learning algorithms, which no longer need a sub-optimal iterative inner-loop adaptation as in the MAML framework. We achieve this goal by 1) replacing the adaptation with a fast-adaptive regularizer in the RKHS; and 2) solving the adaptation analytically based on the NTK theory.
  arXiv  Detail & Related papers  (2021-02-07T20:53:23Z)
• Improving Generalization in Meta-learning via Task Augmentation [69.83677015207527]
  We propose two task augmentation methods, including MetaMix and Channel Shuffle. Both MetaMix and Channel Shuffle outperform state-of-the-art results by a large margin across many datasets.
  arXiv  Detail & Related papers  (2020-07-26T01:50:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.