Template-Driven LLM-Paraphrased Framework for Tabular Math Word Problem Generation

• URL: http://arxiv.org/abs/2412.15594v1
• Date: Fri, 20 Dec 2024 06:34:57 GMT
• Title: Template-Driven LLM-Paraphrased Framework for Tabular Math Word Problem Generation
• Authors: Xiaoqiang Kang, Zimu Wang, Xiaobo Jin, Wei Wang, Kaizhu Huang, Qiufeng Wang,
• Abstract summary: We propose a template-driven LLM-paraphrased (TeLL) framework for generating high-quality TMWP samples with diverse backgrounds and accurate tables, questions, answers, and solutions. Through the proposed framework, we construct a high-quality dataset TabMWP-TeLL by adhering to the question types in the TabMWP dataset.
• Score: 22.6458807254272
• License:
• Abstract: Solving tabular math word problems (TMWPs) has become a critical role in evaluating the mathematical reasoning ability of large language models (LLMs), where large-scale TMWP samples are commonly required for LLM fine-tuning. Since the collection of high-quality TMWP datasets is costly and time-consuming, recent research has concentrated on automatic TMWP generation. However, current generated samples usually suffer from issues of either correctness or diversity. In this paper, we propose a Template-driven LLM-paraphrased (TeLL) framework for generating high-quality TMWP samples with diverse backgrounds and accurate tables, questions, answers, and solutions. To this end, we first extract templates from existing real samples to generate initial problems, ensuring correctness. Then, we adopt an LLM to extend templates and paraphrase problems, obtaining diverse TMWP samples. Furthermore, we find the reasoning annotation is important for solving TMWPs. Therefore, we propose to enrich each solution with illustrative reasoning steps. Through the proposed framework, we construct a high-quality dataset TabMWP-TeLL by adhering to the question types in the TabMWP dataset, and we conduct extensive experiments on a variety of LLMs to demonstrate the effectiveness of TabMWP-TeLL in improving TMWP solving performance. The code and data of this paper are available at: https://github.com/Jason8Kang/TELL.

Related papers

• Neural Topic Modeling with Large Language Models in the Loop [12.142323482188056]
  Large Language Models (LLMs) have demonstrated promising capabilities in topic discovery. We propose LLM-ITL, a novel framework that integrates LLMs with Neural Topic Models (NTMs) Our code and datasets will be available at Github.
  arXiv  Detail & Related papers  (2024-11-13T11:31:02Z)
• SELF-GUIDE: Better Task-Specific Instruction Following via Self-Synthetic Finetuning [70.21358720599821]
  Large language models (LLMs) hold the promise of solving diverse tasks when provided with appropriate natural language prompts. We propose SELF-GUIDE, a multi-stage mechanism in which we synthesize task-specific input-output pairs from the student LLM. We report an absolute improvement of approximately 15% for classification tasks and 18% for generation tasks in the benchmark's metrics.
  arXiv  Detail & Related papers  (2024-07-16T04:41:58Z)
• TACT: Advancing Complex Aggregative Reasoning with Information Extraction Tools [51.576974932743596]
  Large Language Models (LLMs) often do not perform well on queries that require the aggregation of information across texts. TACT contains challenging instructions that demand stitching information scattered across one or more texts. We construct this dataset by leveraging an existing dataset of texts and their associated tables. We demonstrate that all contemporary LLMs perform poorly on this dataset, achieving an accuracy below 38%.
  arXiv  Detail & Related papers  (2024-06-05T20:32:56Z)
• Large Language Models Can Automatically Engineer Features for Few-Shot Tabular Learning [35.03338699349037]
  We propose a novel in-context learning framework, FeatLLM, which employs Large Language Models as feature engineers. FeatLLM generates high-quality rules, significantly (10% on average) outperforming alternatives such as TabLLM and STUNT.
  arXiv  Detail & Related papers  (2024-04-15T06:26:08Z)
• TAT-LLM: A Specialized Language Model for Discrete Reasoning over Tabular and Textual Data [73.29220562541204]
  We consider harnessing the amazing power of language models (LLMs) to solve our task. We develop a TAT-LLM language model by fine-tuning LLaMA 2 with the training data generated automatically from existing expert-annotated datasets.
  arXiv  Detail & Related papers  (2024-01-24T04:28:50Z)
• TAP4LLM: Table Provider on Sampling, Augmenting, and Packing Semi-structured Data for Large Language Model Reasoning [55.33939289989238]
  We propose TAP4LLM as a versatile pre-processor suite for leveraging large language models (LLMs) in table-based tasks effectively. It covers several distinct components: (1) table sampling to decompose large tables into manageable sub-tables based on query semantics, (2) table augmentation to enhance tables with additional knowledge from external sources or models, and (3) table packing & serialization to convert tables into various formats suitable for LLMs' understanding.
  arXiv  Detail & Related papers  (2023-12-14T15:37:04Z)
• Unbiased Math Word Problems Benchmark for Mitigating Solving Bias [72.8677805114825]
  Current solvers exist solving bias which consists of data bias and learning bias due to biased dataset and improper training strategy. Our experiments verify MWP solvers are easy to be biased by the biased training datasets which do not cover diverse questions for each problem narrative of all MWPs. An MWP can be naturally solved by multiple equivalent equations while current datasets take only one of the equivalent equations as ground truth.
  arXiv  Detail & Related papers  (2022-05-17T06:07:04Z)
• MWPToolkit: An Open-Source Framework for Deep Learning-Based Math Word Problem Solvers [29.611442087779896]
  MWPToolkit is the first open-source framework for solving Math Word Problem (MWP) solvers. We implement and compare 17 MWP solvers on 4 widely-used single equation generation benchmarks and 2 multiple equations generation benchmarks.
  arXiv  Detail & Related papers  (2021-09-02T09:18:09Z)
• MWP-BERT: A Strong Baseline for Math Word Problems [47.51572465676904]
  Math word problem (MWP) solving is the task of transforming a sequence of natural language problem descriptions to executable math equations. Although recent sequence modeling MWP solvers have gained credits on the math-text contextual understanding, pre-trained language models (PLM) have not been explored for solving MWP. We introduce MWP-BERT to obtain pre-trained token representations that capture the alignment between text description and mathematical logic.
  arXiv  Detail & Related papers  (2021-07-28T15:28:41Z)
• Are NLP Models really able to Solve Simple Math Word Problems? [7.433931244705934]
  We show that MWP solvers that do not have access to the question asked in the MWP can still solve a large fraction of MWPs. We introduce a challenge dataset, SVAMP, created by applying carefully chosen variations over sampled from existing datasets. The best accuracy achieved by state-of-the-art models is substantially lower on SVAMP, thus showing that much remains to be done even for the simplest of the MWPs.
  arXiv  Detail & Related papers  (2021-03-12T10:23:47Z)
• Semantically-Aligned Universal Tree-Structured Solver for Math Word Problems [129.90766822085132]
  A practical automatic textual math word problems (MWPs) solver should be able to solve various textual MWPs. We propose a simple but efficient method called Universal Expression Tree (UET) to make the first attempt to represent the equations of various MWPs uniformly. Then a semantically-aligned universal tree-structured solver (SAU-r) based on an encoder-decoder framework is proposed to resolve multiple types of MWPs in a unified model.
  arXiv  Detail & Related papers  (2020-10-14T06:27:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.