SAAS: Solving Ability Amplification Strategy for Enhanced Mathematical Reasoning in Large Language Models

• URL: http://arxiv.org/abs/2404.03887v4
• Date: Wed, 02 Oct 2024 11:56:35 GMT
• Title: SAAS: Solving Ability Amplification Strategy for Enhanced Mathematical Reasoning in Large Language Models
• Authors: Hyeonwoo Kim, Gyoungjin Gim, Yungi Kim, Jihoo Kim, Byungju Kim, Wonseok Lee, Chanjun Park,
• Abstract summary: We focus on integrating the Chain-of-Thought (CoT) and the Program-of-Thought (PoT) learning. We propose a sequential learning approach, named SAAS (Solving Ability Amplification Strategy), which strategically transitions from CoT learning to PoT learning.
• Score: 4.090307917818891
• License:
• Abstract: This study presents a novel learning approach designed to enhance both mathematical reasoning and problem-solving abilities of Large Language Models (LLMs). We focus on integrating the Chain-of-Thought (CoT) and the Program-of-Thought (PoT) learning, hypothesizing that prioritizing the learning of mathematical reasoning ability is helpful for the amplification of problem-solving ability. Thus, the initial learning with CoT is essential for solving challenging mathematical problems. To this end, we propose a sequential learning approach, named SAAS (Solving Ability Amplification Strategy), which strategically transitions from CoT learning to PoT learning. Our empirical study, involving an extensive performance comparison using several benchmarks, demonstrates that our SAAS achieves state-of-the-art (SOTA) performance. The results underscore the effectiveness of our sequential learning approach, marking a significant advancement in the field of mathematical reasoning in LLMs.

Related papers

• Inference-Time Computations for LLM Reasoning and Planning: A Benchmark and Insights [49.42133807824413]
  We examine the reasoning and planning capabilities of large language models (LLMs) in solving complex tasks. Recent advances in inference-time techniques demonstrate the potential to enhance LLM reasoning without additional training. OpenAI's o1 model shows promising performance through its novel use of multi-step reasoning and verification.
  arXiv  Detail & Related papers  (2025-02-18T04:11:29Z)
• Teaching LLMs According to Their Aptitude: Adaptive Reasoning for Mathematical Problem Solving [55.895917967408586]
  Existing approaches to mathematical reasoning with large language models rely on Chain-of-Thought (CoT) for generalizability or Tool-Integrated Reasoning (TIR) for precise computation. We propose TATA (Teaching LLMs According to Their Aptitude), an adaptive framework that enables LLMs to personalize their reasoning strategy spontaneously.
  arXiv  Detail & Related papers  (2025-02-17T16:56:23Z)
• Advancing Math Reasoning in Language Models: The Impact of Problem-Solving Data, Data Synthesis Methods, and Training Stages [13.377908992869814]
  Problem-solving data significantly enhances the model's mathematical capabilities compared to general mathematical corpora. We identify effective data synthesis methods, demonstrating that the tutorship amplification synthesis method achieves the best performance.
  arXiv  Detail & Related papers  (2025-01-23T12:14:57Z)
• LLM Reasoning Engine: Specialized Training for Enhanced Mathematical Reasoning [7.512199306943756]
  We present a novel method to enhance Large Language Models' capabilities in mathematical reasoning tasks. Motivated by the need to bridge this gap, our approach incorporates a question paraphrase strategy. specialized training objectives are employed to guide the model's learning process.
  arXiv  Detail & Related papers  (2024-12-28T17:48:33Z)
• Task Arithmetic Through The Lens Of One-Shot Federated Learning [3.8230727103887943]
  Task Arithmetic is a model merging technique that enables the combination of multiple models' capabilities into a single model. We show that Task Arithmetic is mathematically equivalent to the commonly used algorithm in Federated Learning. We adapt several algorithms from Federated Learning to improve the effectiveness of Task Arithmetic.
  arXiv  Detail & Related papers  (2024-11-27T18:53:41Z)
• BloomWise: Enhancing Problem-Solving capabilities of Large Language Models using Bloom's-Taxonomy-Inspired Prompts [59.83547898874152]
  We introduce BloomWise, a new prompting technique, inspired by Bloom's taxonomy, to improve the performance of Large Language Models (LLMs) The decision regarding the need to employ more sophisticated cognitive skills is based on self-evaluation performed by the LLM. In extensive experiments across 4 popular math reasoning datasets, we have demonstrated the effectiveness of our proposed approach.
  arXiv  Detail & Related papers  (2024-10-05T09:27:52Z)
• Benchmarking Large Language Models for Math Reasoning Tasks [12.91916443702145]
  We compare seven state-of-the-art in-context learning algorithms for mathematical problem solving across five widely used mathematical datasets on four powerful foundation models. Our results indicate that larger foundation models like GPT-4o and LLaMA 3-70B can solve mathematical reasoning independently from the concrete prompting strategy. We open-source our benchmark code to support the integration of additional models in future research.
  arXiv  Detail & Related papers  (2024-08-20T13:34:17Z)
• SEGO: Sequential Subgoal Optimization for Mathematical Problem-Solving [64.38649623473626]
  Large Language Models (LLMs) have driven substantial progress in artificial intelligence. We propose a novel framework called textbfSEquential subtextbfGoal textbfOptimization (SEGO) to enhance LLMs' ability to solve mathematical problems.
  arXiv  Detail & Related papers  (2023-10-19T17:56:40Z)
• Auxiliary Learning as an Asymmetric Bargaining Game [50.826710465264505]
  We propose a novel approach, named AuxiNash, for balancing tasks in auxiliary learning. We describe an efficient procedure for learning the bargaining power of tasks based on their contribution to the performance of the main task. We evaluate AuxiNash on multiple multi-task benchmarks and find that it consistently outperforms competing methods.
  arXiv  Detail & Related papers  (2023-01-31T09:41:39Z)
• Importance Weighted Policy Learning and Adaptation [89.46467771037054]
  We study a complementary approach which is conceptually simple, general, modular and built on top of recent improvements in off-policy learning. The framework is inspired by ideas from the probabilistic inference literature and combines robust off-policy learning with a behavior prior. Our approach achieves competitive adaptation performance on hold-out tasks compared to meta reinforcement learning baselines and can scale to complex sparse-reward scenarios.
  arXiv  Detail & Related papers  (2020-09-10T14:16:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.