MATHWELL: Generating Educational Math Word Problems Using Teacher Annotations

• URL: http://arxiv.org/abs/2402.15861v5
• Date: Fri, 27 Sep 2024 11:28:50 GMT
• Title: MATHWELL: Generating Educational Math Word Problems Using Teacher Annotations
• Authors: Bryan R Christ, Jonathan Kropko, Thomas Hartvigsen,
• Abstract summary: We propose that language models have potential to support K-8 math education by automatically generating word problems. Our model, MATHWELL, is the first K-8 word problem generator targeted at educational appropriateness.
• Score: 11.267553596118743
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Math word problems are critical K-8 educational tools, but writing them is time consuming and requires extensive expertise. To be educational, problems must be solvable, have accurate answers, and, most importantly, be educationally appropriate. We propose that language models have potential to support K-8 math education by automatically generating word problems. However, evaluating educational appropriateness is hard to quantify. We fill this gap by having teachers evaluate problems generated by LLMs, who find existing models and data often fail to be educationally appropriate. We then explore automatically generating educational word problems, ultimately using our expert annotations to finetune a 70B language model. Our model, MATHWELL, is the first K-8 word problem generator targeted at educational appropriateness. Further expert studies find MATHWELL generates problems far more solvable, accurate, and appropriate than public models. MATHWELL also matches GPT-4's problem quality while attaining more appropriate reading levels for K-8 students and avoiding generating harmful questions.

Related papers

• Mathfish: Evaluating Language Model Math Reasoning via Grounding in Educational Curricula [25.549869705051606]
  We investigate whether language models' (LMs) mathematical abilities can discern skills and concepts enabled by math content. We develop two tasks for evaluating LMs' abilities to assess math problems. We find that LMs struggle to tag and verify standards linked to problems, and instead predict labels that are close to ground truth, but differ in subtle ways.
  arXiv  Detail & Related papers  (2024-08-08T05:28:34Z)
• Physics of Language Models: Part 2.1, Grade-School Math and the Hidden Reasoning Process [47.753284211200665]
  Recent advances in language models have demonstrated their capability to solve mathematical reasoning problems. Our study uncovers many hidden mechanisms by which language models solve mathematical questions.
  arXiv  Detail & Related papers  (2024-07-29T17:52:40Z)
• Stepwise Verification and Remediation of Student Reasoning Errors with Large Language Model Tutors [78.53699244846285]
  Large language models (LLMs) present an opportunity to scale high-quality personalized education to all. LLMs struggle to precisely detect student's errors and tailor their feedback to these errors. Inspired by real-world teaching practice where teachers identify student errors and customize their response based on them, we focus on verifying student solutions.
  arXiv  Detail & Related papers  (2024-07-12T10:11:40Z)
• MathCAMPS: Fine-grained Synthesis of Mathematical Problems From Human Curricula [33.5782208232163]
  We propose Math CAMPS: a method to synthesize high-quality mathematical problems at scale. We encode each standard in a formal grammar, allowing us to sample diverse symbolic problems and their answers. We derive follow-up questions from symbolic structures and convert them into follow-up word problems.
  arXiv  Detail & Related papers  (2024-07-01T01:56:28Z)
• DiVERT: Distractor Generation with Variational Errors Represented as Text for Math Multiple-choice Questions [42.148511874019256]
  We introduce DiVERT, a novel variational approach that learns an interpretable representation of errors behind distractors in math multiple-choice questions (MCQs) We show that DiVERT, despite using a base open-source LLM with 7B parameters, outperforms state-of-the-art approaches using GPT-4o on downstream distractor generation. We also conduct a human evaluation with math educators and find that DiVERT leads to error labels that are of comparable quality to human-authored ones.
  arXiv  Detail & Related papers  (2024-06-27T17:37:31Z)
• Language Models as Science Tutors [79.73256703631492]
  We introduce TutorEval and TutorChat to measure real-life usability of LMs as scientific assistants. We show that fine-tuning base models with existing dialogue datasets leads to poor performance on TutorEval. We use TutorChat to fine-tune Llemma models with 7B and 34B parameters. These LM tutors specialized in math have a 32K-token context window, and they excel at TutorEval while performing strongly on GSM8K and MATH.
  arXiv  Detail & Related papers  (2024-02-16T22:24:13Z)
• MathDial: A Dialogue Tutoring Dataset with Rich Pedagogical Properties Grounded in Math Reasoning Problems [74.73881579517055]
  We propose a framework to generate such dialogues by pairing human teachers with a Large Language Model prompted to represent common student errors. We describe how we use this framework to collect MathDial, a dataset of 3k one-to-one teacher-student tutoring dialogues.
  arXiv  Detail & Related papers  (2023-05-23T21:44:56Z)
• Automatic Generation of Socratic Subquestions for Teaching Math Word Problems [16.97827669744673]
  We explore the ability of large language models (LMs) in generating sequential questions for guiding math word problem-solving. On both automatic and human quality evaluations, we find that LMs constrained with desirable question properties generate superior questions. Results suggest that the difficulty level of problems plays an important role in determining whether questioning improves or hinders human performance.
  arXiv  Detail & Related papers  (2022-11-23T10:40:22Z)
• Why are NLP Models Fumbling at Elementary Math? A Survey of Deep Learning based Word Problem Solvers [7.299537282917047]
  We critically examine the various models that have been developed for solving word problems. We take a step back and analyse why, in spite of this abundance in scholarly interest, the predominantly used experiment and dataset designs continue to be a stumbling block.
  arXiv  Detail & Related papers  (2022-05-31T10:51:25Z)
• Measuring Mathematical Problem Solving With the MATH Dataset [55.4376028963537]
  We introduce MATH, a dataset of 12,500 challenging competition mathematics problems. Each problem has a full step-by-step solution which can be used to teach models to generate answer derivations and explanations. We also contribute a large auxiliary pretraining dataset which helps teach models the fundamentals of mathematics.
  arXiv  Detail & Related papers  (2021-03-05T18:59:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.