Techniques to Improve Neural Math Word Problem Solvers
- URL: http://arxiv.org/abs/2302.03145v1
- Date: Mon, 6 Feb 2023 22:41:51 GMT
- Title: Techniques to Improve Neural Math Word Problem Solvers
- Authors: Youyuan Zhang
- Abstract summary: Recent neural-based approaches mainly encode the problem text using a language model and decode a mathematical expression over quantities and operators iteratively.
We propose a new encoder-decoder architecture that fully leverages the question text and preserves step-wise commutative law.
Experiments on four established benchmarks demonstrate that our framework outperforms state-of-the-art neural MWP solvers.
- Score: 0.0
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Developing automatic Math Word Problem (MWP) solvers is a challenging task
that demands the ability of understanding and mathematical reasoning over the
natural language. Recent neural-based approaches mainly encode the problem text
using a language model and decode a mathematical expression over quantities and
operators iteratively. Note the problem text of a MWP consists of a context
part and a question part, a recent work finds these neural solvers may only
perform shallow pattern matching between the context text and the golden
expression, where question text is not well used. Meanwhile, existing decoding
processes fail to enforce the mathematical laws into the design, where the
representations for mathematical equivalent expressions are different. To
address these two issues, we propose a new encoder-decoder architecture that
fully leverages the question text and preserves step-wise commutative law.
Besides generating quantity embeddings, our encoder further encodes the
question text and uses it to guide the decoding process. At each step, our
decoder uses Deep Sets to compute expression representations so that these
embeddings are invariant under any permutation of quantities. Experiments on
four established benchmarks demonstrate that our framework outperforms
state-of-the-art neural MWP solvers, showing the effectiveness of our
techniques. We also conduct a detailed analysis of the results to show the
limitations of our approach and further discuss the potential future work. Code
is available at https://github.com/sophistz/Question-Aware-Deductive-MWP.
Related papers
- Solving Math Word Problems with Reexamination [27.80592576792461]
We propose a pseudo-dual (PseDual) learning scheme to model such process, which is model-agnostic.
The pseudo-dual task is specifically defined as filling the numbers in the expression back into the original word problem with numbers masked.
Our pseudo-dual learning scheme has been tested and proven effective when being equipped in several representative MWP solvers through empirical studies.
arXiv Detail & Related papers (2023-10-14T14:23:44Z) - Math Word Problem Solving by Generating Linguistic Variants of Problem
Statements [1.742186232261139]
We propose a framework for MWP solvers based on the generation of linguistic variants of the problem text.
The approach involves solving each of the variant problems and electing the predicted expression with the majority of the votes.
We show that training on linguistic variants of problem statements and voting on candidate predictions improve the mathematical reasoning and robustness of the model.
arXiv Detail & Related papers (2023-06-24T08:27:39Z) - Momentum Decoding: Open-ended Text Generation As Graph Exploration [49.812280360794894]
Open-ended text generation with autoregressive language models (LMs) is one of the core tasks in natural language processing.
We formulate open-ended text generation from a new perspective, i.e., we view it as an exploration process within a directed graph.
We propose a novel decoding method -- textitmomentum decoding -- which encourages the LM to explore new nodes outside the current graph.
arXiv Detail & Related papers (2022-12-05T11:16:47Z) - Revisiting the Roles of "Text" in Text Games [102.22750109468652]
This paper investigates the roles of text in the face of different reinforcement learning challenges.
We propose a simple scheme to extract relevant contextual information into an approximate state hash.
Such a lightweight plug-in achieves competitive performance with state-of-the-art text agents.
arXiv Detail & Related papers (2022-10-15T21:52:39Z) - JiuZhang: A Chinese Pre-trained Language Model for Mathematical Problem
Understanding [74.12405417718054]
This paper aims to advance the mathematical intelligence of machines by presenting the first Chinese mathematical pre-trained language model(PLM)
Unlike other standard NLP tasks, mathematical texts are difficult to understand, since they involve mathematical terminology, symbols and formulas in the problem statement.
We design a novel curriculum pre-training approach for improving the learning of mathematical PLMs, consisting of both basic and advanced courses.
arXiv Detail & Related papers (2022-06-13T17:03:52Z) - Tackling Math Word Problems with Fine-to-Coarse Abstracting and
Reasoning [22.127301797950572]
We propose to model a math word problem in a fine-to-coarse manner to capture both the local fine-grained information and the global logical structure of it.
Our model is naturally sensitive to local variations and can better generalize to unseen problem types.
arXiv Detail & Related papers (2022-05-17T12:14:44Z) - MWP-BERT: A Strong Baseline for Math Word Problems [47.51572465676904]
Math word problem (MWP) solving is the task of transforming a sequence of natural language problem descriptions to executable math equations.
Although recent sequence modeling MWP solvers have gained credits on the math-text contextual understanding, pre-trained language models (PLM) have not been explored for solving MWP.
We introduce MWP-BERT to obtain pre-trained token representations that capture the alignment between text description and mathematical logic.
arXiv Detail & Related papers (2021-07-28T15:28:41Z) - Neural-Symbolic Solver for Math Word Problems with Auxiliary Tasks [130.70449023574537]
Our NS-r consists of a problem reader to encode problems, a programmer to generate symbolic equations, and a symbolic executor to obtain answers.
Along with target expression supervision, our solver is also optimized via 4 new auxiliary objectives to enforce different symbolic reasoning.
arXiv Detail & Related papers (2021-07-03T13:14:58Z) - Generating Math Word Problems from Equations with Topic Controlling and
Commonsense Enforcement [11.459200644989227]
We present a novel equation-to-problem text generation model.
In our model, 1) we propose a flexible scheme to effectively encode math equations, we then enhance the equation encoder by a Varitional Autoen-coder (VAE)
arXiv Detail & Related papers (2020-12-14T10:02:11Z) - Machine Number Sense: A Dataset of Visual Arithmetic Problems for
Abstract and Relational Reasoning [95.18337034090648]
We propose a dataset, Machine Number Sense (MNS), consisting of visual arithmetic problems automatically generated using a grammar model--And-Or Graph (AOG)
These visual arithmetic problems are in the form of geometric figures.
We benchmark the MNS dataset using four predominant neural network models as baselines in this visual reasoning task.
arXiv Detail & Related papers (2020-04-25T17:14:58Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.