Explaining Math Word Problem Solvers

• URL: http://arxiv.org/abs/2307.13128v1
• Date: Mon, 24 Jul 2023 21:05:47 GMT
• Title: Explaining Math Word Problem Solvers
• Authors: Abby Newcomb and Jugal Kalita
• Abstract summary: We investigate what information math word problem solvers use to generate solutions. Our results show that the model is not sensitive to the removal of many words from the input and can still find a correct answer when given a nonsense question. This indicates that automatic solvers do not follow the semantic logic of math word problems, and may be overfitting to the presence of specific words.
• Score: 2.792030485253753
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Automated math word problem solvers based on neural networks have successfully managed to obtain 70-80\% accuracy in solving arithmetic word problems. However, it has been shown that these solvers may rely on superficial patterns to obtain their equations. In order to determine what information math word problem solvers use to generate solutions, we remove parts of the input and measure the model's performance on the perturbed dataset. Our results show that the model is not sensitive to the removal of many words from the input and can still manage to find a correct answer when given a nonsense question. This indicates that automatic solvers do not follow the semantic logic of math word problems, and may be overfitting to the presence of specific words.

Related papers

• Do Language Models Exhibit the Same Cognitive Biases in Problem Solving as Human Learners? [140.9751389452011]
  We study the biases of large language models (LLMs) in relation to those known in children when solving arithmetic word problems. We generate a novel set of word problems for each of these tests, using a neuro-symbolic approach that enables fine-grained control over the problem features.
  arXiv  Detail & Related papers  (2024-01-31T18:48:20Z)
• Verb Categorisation for Hindi Word Problem Solving [4.926283917321646]
  We have built a Hindi arithmetic word problem solver which makes use of verbs. We propose a rule-based solver that uses verb categorisation to identify operations in a word problem and generate answers for it.
  arXiv  Detail & Related papers  (2023-12-18T17:55:05Z)
• Solving Math Word Problems by Combining Language Models With Symbolic Solvers [28.010617102877923]
  Large language models (LLMs) can be combined with external tools to perform complex reasoning and calculation. We propose an approach that combines an LLM that can incrementally formalize word problems as a set of variables and equations with an external symbolic solver. Our approach achieves comparable accuracy to the original PAL on the GSM8K benchmark of math word problems and outperforms PAL by an absolute 20% on ALGEBRA.
  arXiv  Detail & Related papers  (2023-04-16T04:16:06Z)
• Limits of an AI program for solving college math problems [0.0]
  A neural network solves, explains, and generates university math problems by program synthesis and few-shot learning at human level. The system they describe is indeed impressive; however, the above description is very much overstated. The work of solving the problems is done, not by a neural network, but by the symbolic algebra package Sympy.
  arXiv  Detail & Related papers  (2022-08-14T20:10:14Z)
• Tackling Math Word Problems with Fine-to-Coarse Abstracting and Reasoning [22.127301797950572]
  We propose to model a math word problem in a fine-to-coarse manner to capture both the local fine-grained information and the global logical structure of it. Our model is naturally sensitive to local variations and can better generalize to unseen problem types.
  arXiv  Detail & Related papers  (2022-05-17T12:14:44Z)
• A Mutual Information Maximization Approach for the Spurious Solution Problem in Weakly Supervised Question Answering [60.768146126094955]
  Weakly supervised question answering usually has only the final answers as supervision signals. There may exist many spurious solutions that coincidentally derive the correct answer, but training on such solutions can hurt model performance. We propose to explicitly exploit such semantic correlations by maximizing the mutual information between question-answer pairs and predicted solutions.
  arXiv  Detail & Related papers  (2021-06-14T05:47:41Z)
• Measuring Mathematical Problem Solving With the MATH Dataset [55.4376028963537]
  We introduce MATH, a dataset of 12,500 challenging competition mathematics problems. Each problem has a full step-by-step solution which can be used to teach models to generate answer derivations and explanations. We also contribute a large auxiliary pretraining dataset which helps teach models the fundamentals of mathematics.
  arXiv  Detail & Related papers  (2021-03-05T18:59:39Z)
• SMART: A Situation Model for Algebra Story Problems via Attributed Grammar [74.1315776256292]
  We introduce the concept of a emphsituation model, which originates from psychology studies to represent the mental states of humans in problem-solving. We show that the proposed model outperforms all previous neural solvers by a large margin while preserving much better interpretability.
  arXiv  Detail & Related papers  (2020-12-27T21:03:40Z)
• Learning by Fixing: Solving Math Word Problems with Weak Supervision [70.62896781438694]
  Previous neural solvers of math word problems (MWPs) are learned with full supervision and fail to generate diverse solutions. We introduce a textitweakly-supervised paradigm for learning MWPs. Our method only requires the annotations of the final answers and can generate various solutions for a single problem.
  arXiv  Detail & Related papers  (2020-12-19T03:10:21Z)
• Reverse Operation based Data Augmentation for Solving Math Word Problems [37.26159426631031]
  Recent models have reached their performance bottleneck and require more high-quality data for training. We propose a novel data augmentation method that reverses the mathematical logic of math word problems. We apply the augmented data on two SOTA math word problem solving models and compare our results with a strong data augmentation baseline.
  arXiv  Detail & Related papers  (2020-10-04T11:59:59Z)
• Machine Number Sense: A Dataset of Visual Arithmetic Problems for Abstract and Relational Reasoning [95.18337034090648]
  We propose a dataset, Machine Number Sense (MNS), consisting of visual arithmetic problems automatically generated using a grammar model--And-Or Graph (AOG) These visual arithmetic problems are in the form of geometric figures. We benchmark the MNS dataset using four predominant neural network models as baselines in this visual reasoning task.
  arXiv  Detail & Related papers  (2020-04-25T17:14:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.