ATHENA: Mathematical Reasoning with Thought Expansion
- URL: http://arxiv.org/abs/2311.01036v1
- Date: Thu, 2 Nov 2023 07:03:25 GMT
- Title: ATHENA: Mathematical Reasoning with Thought Expansion
- Authors: JB. Kim, Hazel Kim, Joonghyuk Hahn, Yo-Sub Han
- Abstract summary: We introduce Attention-based THought Expansion Network Architecture (ATHENA) to tackle the challenges of real-world practices.
A thought expansion recurrently generates the candidates carrying the thoughts of possible math expressions driven from the previous step.
- Score: 3.3727470465639833
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Solving math word problems depends on how to articulate the problems, the
lens through which models view human linguistic expressions. Real-world
settings count on such a method even more due to the diverse practices of the
same mathematical operations. Earlier works constrain available thinking
processes by limited prediction strategies without considering their
significance in acquiring mathematical knowledge. We introduce Attention-based
THought Expansion Network Architecture (ATHENA) to tackle the challenges of
real-world practices by mimicking human thought expansion mechanisms in the
form of neural network propagation. A thought expansion recurrently generates
the candidates carrying the thoughts of possible math expressions driven from
the previous step and yields reasonable thoughts by selecting the valid
pathways to the goal. Our experiments show that ATHENA achieves a new
state-of-the-art stage toward the ideal model that is compelling in variant
questions even when the informativeness in training examples is restricted.
Related papers
- Learn Beyond The Answer: Training Language Models with Reflection for Mathematical Reasoning [48.923678251844194]
Supervised fine-tuning enhances the problem-solving abilities of language models across various mathematical reasoning tasks.
Our work introduces a novel technique aimed at cultivating a deeper understanding of the training problems at hand.
We propose reflective augmentation, a method that embeds problem reflection into each training instance.
arXiv Detail & Related papers (2024-06-17T19:42:22Z) - Brain-Inspired Two-Stage Approach: Enhancing Mathematical Reasoning by
Imitating Human Thought Processes [6.512667145063511]
We propose a novel approach, named Brain, to imitate human thought processes to enhance mathematical reasoning abilities.
First, we achieve SOTA performance in comparison with Code LLaMA 7B based models through this method.
Secondly, we find that plans can be explicitly extracted from natural language, code, or formal language.
arXiv Detail & Related papers (2024-02-23T17:40:31Z) - Large Language Models as Analogical Reasoners [155.9617224350088]
Chain-of-thought (CoT) prompting for language models demonstrates impressive performance across reasoning tasks.
We introduce a new prompting approach, analogical prompting, designed to automatically guide the reasoning process of large language models.
arXiv Detail & Related papers (2023-10-03T00:57:26Z) - The Art of SOCRATIC QUESTIONING: Recursive Thinking with Large Language
Models [45.01562498702836]
Chain-of-Thought (CoT) prompting enables large language models to solve complex reasoning problems by generating intermediate steps.
We propose SOCRATIC QUESTIONING, a divide-and-conquer style algorithm that mimics the recursive thinking process.
arXiv Detail & Related papers (2023-05-24T10:36:14Z) - Learning to solve arithmetic problems with a virtual abacus [0.35911228556176483]
We introduce a deep reinforcement learning framework that allows to simulate how cognitive agents could learn to solve arithmetic problems.
The proposed model successfully learns to perform multi-digit additions and subtractions, achieving an error rate below 1%.
We analyze the most common error patterns to better understand the limitations and biases resulting from our design choices.
arXiv Detail & Related papers (2023-01-17T13:25:52Z) - A Survey of Deep Learning for Mathematical Reasoning [71.88150173381153]
We review the key tasks, datasets, and methods at the intersection of mathematical reasoning and deep learning over the past decade.
Recent advances in large-scale neural language models have opened up new benchmarks and opportunities to use deep learning for mathematical reasoning.
arXiv Detail & Related papers (2022-12-20T18:46:16Z) - Chaining Simultaneous Thoughts for Numerical Reasoning [92.2007997126144]
numerical reasoning over text should be an essential skill of AI systems.
Previous work focused on modeling the structures of equations, and has proposed various structured decoders.
We propose CANTOR, a numerical reasoner that models reasoning steps using a directed acyclic graph.
arXiv Detail & Related papers (2022-11-29T18:52:06Z) - Peano: Learning Formal Mathematical Reasoning [35.086032962873226]
General mathematical reasoning is computationally undecidable, but humans routinely solve new problems.
We posit that central to both puzzles is the structure of procedural abstractions underlying mathematics.
We explore this idea in a case study on 5 sections of beginning algebra on the Khan Academy platform.
arXiv Detail & Related papers (2022-11-29T01:42:26Z) - SMART: A Situation Model for Algebra Story Problems via Attributed
Grammar [74.1315776256292]
We introduce the concept of a emphsituation model, which originates from psychology studies to represent the mental states of humans in problem-solving.
We show that the proposed model outperforms all previous neural solvers by a large margin while preserving much better interpretability.
arXiv Detail & Related papers (2020-12-27T21:03:40Z) - Machine Number Sense: A Dataset of Visual Arithmetic Problems for
Abstract and Relational Reasoning [95.18337034090648]
We propose a dataset, Machine Number Sense (MNS), consisting of visual arithmetic problems automatically generated using a grammar model--And-Or Graph (AOG)
These visual arithmetic problems are in the form of geometric figures.
We benchmark the MNS dataset using four predominant neural network models as baselines in this visual reasoning task.
arXiv Detail & Related papers (2020-04-25T17:14:58Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.