Learn Beyond The Answer: Training Language Models with Reflection for Mathematical Reasoning

• URL: http://arxiv.org/abs/2406.12050v3
• Date: Sat, 05 Oct 2024 21:30:51 GMT
• Title: Learn Beyond The Answer: Training Language Models with Reflection for Mathematical Reasoning
• Authors: Zhihan Zhang, Tao Ge, Zhenwen Liang, Wenhao Yu, Dian Yu, Mengzhao Jia, Dong Yu, Meng Jiang,
• Abstract summary: Supervised fine-tuning enhances the problem-solving abilities of language models across various mathematical reasoning tasks. Our work introduces a novel technique aimed at cultivating a deeper understanding of the training problems at hand. We propose reflective augmentation, a method that embeds problem reflection into each training instance.
• Score: 59.98430756337374
• License:
• Abstract: Supervised fine-tuning enhances the problem-solving abilities of language models across various mathematical reasoning tasks. To maximize such benefits, existing research focuses on broadening the training set with various data augmentation techniques, which is effective for standard single-round question-answering settings. Our work introduces a novel technique aimed at cultivating a deeper understanding of the training problems at hand, enhancing performance not only in standard settings but also in more complex scenarios that require reflective thinking. Specifically, we propose reflective augmentation, a method that embeds problem reflection into each training instance. It trains the model to consider alternative perspectives and engage with abstractions and analogies, thereby fostering a thorough comprehension through reflective reasoning. Extensive experiments validate the achievement of our aim, underscoring the unique advantages of our method and its complementary nature relative to existing augmentation techniques.

Related papers

• BloomWise: Enhancing Problem-Solving capabilities of Large Language Models using Bloom's-Taxonomy-Inspired Prompts [59.83547898874152]
  We introduce BloomWise, a new prompting technique, inspired by Bloom's taxonomy, to improve the performance of Large Language Models (LLMs) The decision regarding the need to employ more sophisticated cognitive skills is based on self-evaluation performed by the LLM. In extensive experiments across 4 popular math reasoning datasets, we have demonstrated the effectiveness of our proposed approach.
  arXiv  Detail & Related papers  (2024-10-05T09:27:52Z)
• Model-Based Transfer Learning for Contextual Reinforcement Learning [5.5597941107270215]
  We show how to systematically select good tasks to train, maximizing overall performance across a range of tasks. Key idea behind our approach is to explicitly model the performance loss incurred by transferring a trained model. We experimentally validate our methods using urban traffic and standard control benchmarks.
  arXiv  Detail & Related papers  (2024-08-08T14:46:01Z)
• Inference Optimizations for Large Language Models: Effects, Challenges, and Practical Considerations [0.0]
  Large language models are ubiquitous in natural language processing because they can adapt to new tasks without retraining. This literature review focuses on various techniques for reducing resource requirements and compressing large language models.
  arXiv  Detail & Related papers  (2024-08-06T12:07:32Z)
• RLIF: Interactive Imitation Learning as Reinforcement Learning [56.997263135104504]
  We show how off-policy reinforcement learning can enable improved performance under assumptions that are similar but potentially even more practical than those of interactive imitation learning. Our proposed method uses reinforcement learning with user intervention signals themselves as rewards. This relaxes the assumption that intervening experts in interactive imitation learning should be near-optimal and enables the algorithm to learn behaviors that improve over the potential suboptimal human expert.
  arXiv  Detail & Related papers  (2023-11-21T21:05:21Z)
• Deep Generative Models for Decision-Making and Control [4.238809918521607]
  The dual purpose of this thesis is to study the reasons for these shortcomings and to propose solutions for the uncovered problems. We highlight how inference techniques from the contemporary generative modeling toolbox, including beam search, can be reinterpreted as viable planning strategies for reinforcement learning problems.
  arXiv  Detail & Related papers  (2023-06-15T01:54:30Z)
• Rethinking Learning Dynamics in RL using Adversarial Networks [79.56118674435844]
  We present a learning mechanism for reinforcement learning of closely related skills parameterized via a skill embedding space. The main contribution of our work is to formulate an adversarial training regime for reinforcement learning with the help of entropy-regularized policy gradient formulation.
  arXiv  Detail & Related papers  (2022-01-27T19:51:09Z)
• Importance Weighted Policy Learning and Adaptation [89.46467771037054]
  We study a complementary approach which is conceptually simple, general, modular and built on top of recent improvements in off-policy learning. The framework is inspired by ideas from the probabilistic inference literature and combines robust off-policy learning with a behavior prior. Our approach achieves competitive adaptation performance on hold-out tasks compared to meta reinforcement learning baselines and can scale to complex sparse-reward scenarios.
  arXiv  Detail & Related papers  (2020-09-10T14:16:58Z)
• A Competence-aware Curriculum for Visual Concepts Learning via Question Answering [95.35905804211698]
  We propose a competence-aware curriculum for visual concept learning in a question-answering manner. We design a neural-symbolic concept learner for learning the visual concepts and a multi-dimensional Item Response Theory (mIRT) model for guiding the learning process. Experimental results on CLEVR show that with a competence-aware curriculum, the proposed method achieves state-of-the-art performances.
  arXiv  Detail & Related papers  (2020-07-03T05:08:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.