Can AI Tools Transform Low-Demand Math Tasks? An Evaluation of Task Modification Capabilities

• URL: http://arxiv.org/abs/2604.12743v1
• Date: Tue, 14 Apr 2026 13:57:09 GMT
• Title: Can AI Tools Transform Low-Demand Math Tasks? An Evaluation of Task Modification Capabilities
• Abstract summary: This study investigated whether AI tools could successfully upgrade low-cognitive-demand mathematics tasks.<n>On average, AI tools were only moderately successful: tasks were accurately upgraded only 64% of the time.<n> Specialized tools were only moderately more successful than general-purpose tools.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While recent research has explored AI tools' ability to classify the quality of mathematical tasks (arXiv:2603.03512), little is known about their capacity to increase the quality of existing tasks. This study investigated whether AI tools could successfully upgrade low-cognitive-demand mathematics tasks. Eleven tools were tested, including six broadly available, general-purpose AI tools (e.g., ChatGPT and Claude) and five tools specialized for mathematics teachers (e.g., Khanmigo, coteach.ai). Using the Task Analysis Guide framework (Stein & Smith, 1998), we prompted AI tools to modify two different types of low-demand mathematical tasks. The prompting strategy aimed to represent likely approaches taken by knowledgeable teachers, rather than extensive optimization to find a more effective prompt (i.e., an optimistic typical outcome). On average, AI tools were only moderately successful: tasks were accurately upgraded only 64% of the time, with different AI tool performance ranging from quite weak (33%) to broadly successful (88%). Specialized tools were only moderately more successful than general-purpose tools. Failure modes included both "undershooting" (maintaining low cognitive demand) and "overshooting" (elevating tasks to an overly ambitious target category that likely would be rejected by teachers). Interestingly, there was a small negative correlation (r = -.35) between whether a given AI tool was able to correctly classify the cognitive demand of tasks and whether the AI was able to upgrade tasks, showing that the ability to modify tasks (i.e., a generative task) represents a distinct capability from the ability to classify them (i.e., judgement using a rubric). These findings have important implications for understanding AI's potential role in curriculum adaptation and highlight the need for specialized approaches to support teachers in modifying instructional materials.

Related papers

• Temporal Stability and Few-Shot Prompting in Math Task Assessment [0.0]
  This study focused on different AI tools' ability to use the Task Analysis Guide to classify cognitive demand of mathematics tasks.<n>We tested a general-purpose AI tool (Gemini) and an education-specific AI tool (Coteach)<n>Results revealed that newer model versions alone produced mixed effects: Gemini's accuracy remained stable at 58%, while Coteach's accuracy decreased from 75% to 50%.
  arXiv  Detail & Related papers  (2026-05-28T16:10:24Z)
• Baseline Performance of AI Tools in Classifying Cognitive Demand of Mathematical Tasks [0.0]
  We tested eleven AI tools on their ability to categorize mathematics tasks across four levels of cognitive demand.<n>On average, AI tools accurately classified cognitive demand in only 63% of cases.<n>All tools struggled with tasks at the extremes of cognitive demand.
  arXiv  Detail & Related papers  (2026-03-03T20:39:55Z)
• AdaReasoner: Dynamic Tool Orchestration for Iterative Visual Reasoning [66.24374176797075]
  We introduce textbfAdaReasoner, a family of multimodal models that learn tool use as a general reasoning skill rather than as tool-specific or explicitly supervised behavior.<n>AdaReasoner is enabled by (i) a scalable data curation pipeline exposing models to long-horizon, multi-step tool interactions; (ii) Tool-GRPO, a reinforcement learning algorithm that prioritizes tool selection and sequencing based on end-task success; and (iii) an adaptive learning mechanism that dynamically regulates tool usage.
  arXiv  Detail & Related papers  (2026-01-26T16:04:43Z)
• What Work is AI Actually Doing? Uncovering the Drivers of Generative AI Adoption [1.4977849232424492]
  This study investigates which intrinsic task characteristics drive users' decisions to delegate work to AI systems.<n>This research provides the first systematic evidence linking real-world generative AI usage to a comprehensive, multi-dimensional framework of intrinsic task characteristics.
  arXiv  Detail & Related papers  (2025-10-26T19:13:37Z)
• AI Agents as Universal Task Solvers [94.49762121230042]
  We show that the optimal speed-up that a universal solver can achieve using past data is tightly related to their algorithmic information.<n>We argue that the key quantity to optimize when scaling reasoning models is time, whose critical role in learning has so far only been indirectly considered.
  arXiv  Detail & Related papers  (2025-10-14T02:17:54Z)
• Self-Challenging Language Model Agents [98.62637336505242]
  We propose the Self-Challenging framework for training an agent on high-quality tasks that are generated by itself.<n>The framework achieves over a two-fold improvement in Llama-3.1-8B-Instruct, despite using only self-generated training data.
  arXiv  Detail & Related papers  (2025-06-02T14:23:33Z)
• General Scales Unlock AI Evaluation with Explanatory and Predictive Power [57.7995945974989]
  benchmarking has guided progress in AI, but it has offered limited explanatory and predictive power for general-purpose AI systems.<n>We introduce general scales for AI evaluation that can explain what common AI benchmarks really measure.<n>Our fully-automated methodology builds on 18 newly-crafted rubrics that place instance demands on general scales that do not saturate.
  arXiv  Detail & Related papers  (2025-03-09T01:13:56Z)
• The Mathematics of Artificial Intelligence [23.03787751696068]
  This overview article highlights the critical role of mathematics in artificial intelligence (AI)<n>It emphasizes that mathematics provides tools to better understand and enhance AI systems.<n> Conversely, AI raises new problems and drives the development of new mathematics at the intersection of various fields.
  arXiv  Detail & Related papers  (2025-01-15T15:00:23Z)
• Measuring Mathematical Problem Solving With the MATH Dataset [55.4376028963537]
  We introduce MATH, a dataset of 12,500 challenging competition mathematics problems. Each problem has a full step-by-step solution which can be used to teach models to generate answer derivations and explanations. We also contribute a large auxiliary pretraining dataset which helps teach models the fundamentals of mathematics.
  arXiv  Detail & Related papers  (2021-03-05T18:59:39Z)
• Automatic Curriculum Learning through Value Disagreement [95.19299356298876]
  Continually solving new, unsolved tasks is the key to learning diverse behaviors. In the multi-task domain, where an agent needs to reach multiple goals, the choice of training goals can largely affect sample efficiency. We propose setting up an automatic curriculum for goals that the agent needs to solve. We evaluate our method across 13 multi-goal robotic tasks and 5 navigation tasks, and demonstrate performance gains over current state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-06-17T03:58:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.