SafeMath: Inference-time Safety improves Math Accuracy

• URL: http://arxiv.org/abs/2603.25201v1
• Date: Thu, 26 Mar 2026 09:06:46 GMT
• Title: SafeMath: Inference-time Safety improves Math Accuracy
• Abstract summary: We show that math questions, particularly those framed as natural language narratives, can serve as a subtle medium for propagating biased, unethical, or psychologically harmful content.<n>We propose SafeMath -- a safety alignment technique that reduces harmful outputs while maintaining, and in some cases improving, mathematical reasoning performance.
• Score: 7.757751847128239
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent research points toward LLMs being manipulated through adversarial and seemingly benign inputs, resulting in harmful, biased, or policy-violating outputs. In this paper, we study an underexplored issue concerning harmful and toxic mathematical word problems. We show that math questions, particularly those framed as natural language narratives, can serve as a subtle medium for propagating biased, unethical, or psychologically harmful content, with heightened risks in educational settings involving children. To support a systematic study of this phenomenon, we introduce ToxicGSM, a dataset of 1.9k arithmetic problems in which harmful or sensitive context is embedded while preserving mathematically well-defined reasoning tasks. Using this dataset, we audit the behaviour of existing LLMs and analyse the trade-offs between safety enforcement and mathematical correctness. We further propose SafeMath -- a safety alignment technique that reduces harmful outputs while maintaining, and in some cases improving, mathematical reasoning performance. Our results highlight the importance of disentangling linguistic harm from math reasoning and demonstrate that effective safety alignment need not come at the cost of accuracy. We release the source code and dataset at https://github.com/Swagnick99/SafeMath/tree/main.

Related papers

• Exposing LLM Safety Gaps Through Mathematical Encoding:New Attacks and Systematic Analysis [8.82937816029695]
  Large language models (LLMs) employ safety mechanisms to prevent harmful outputs.<n>We show that encoding harmful prompts as coherent mathematical problems bypasses these filters at high rates.<n>We introduce a novel Formal Logic encoding that achieves attack success comparable to Set Theory.
  arXiv  Detail & Related papers  (2026-05-05T07:25:33Z)
• HorizonMath: Measuring AI Progress Toward Mathematical Discovery with Automatic Verification [54.06301039725887]
  We introduce HorizonMath, a benchmark of over 100 predominantly unsolved problems spanning 8 domains in computational and applied mathematics.<n>Our benchmark targets a class of problems where discovery is hard, requiring meaningful mathematical insight, but verification is computationally efficient and simple.
  arXiv  Detail & Related papers  (2026-03-16T17:59:53Z)
• MSCR: Exploring the Vulnerability of LLMs' Mathematical Reasoning Abilities Using Multi-Source Candidate Replacement [28.350371282079845]
  We propose MSCR, an automated adversarial attack method based on multi-source candidate replacement.<n>We conduct large-scale experiments on LLMs using the GSM8K and MATH500 benchmarks.<n>Even a slight perturbation involving only a single word can significantly reduce the accuracy of all models.
  arXiv  Detail & Related papers  (2025-11-11T09:56:19Z)
• Guardians and Offenders: A Survey on Harmful Content Generation and Safety Mitigation of LLM [13.066526969147501]
  Large Language Models (LLMs) have revolutionized content creation across digital platforms.<n>LLMs enable beneficial applications such as content generation, question and answering (Q&A), programming, and code reasoning.<n>They also pose serious risks by inadvertently or intentionally producing toxic, offensive, or biased content.
  arXiv  Detail & Related papers  (2025-08-07T18:42:16Z)
• Evaluating Intermediate Reasoning of Code-Assisted Large Language Models for Mathematics [15.695635219034328]
  We conduct an in-depth analysis of code-assisted LLMs generated programs in response to math reasoning tasks.<n>Our findings show that the capabilities of models significantly impact the logic implemented to solve the problem.
  arXiv  Detail & Related papers  (2025-04-24T15:34:24Z)
• LLM Critics Help Catch Bugs in Mathematics: Towards a Better Mathematical Verifier with Natural Language Feedback [71.95402654982095]
  We propose Math-Minos, a natural language feedback-enhanced verifier. Our experiments reveal that a small set of natural language feedback can significantly boost the performance of the verifier.
  arXiv  Detail & Related papers  (2024-06-20T06:42:27Z)
• ALERT: A Comprehensive Benchmark for Assessing Large Language Models' Safety through Red Teaming [64.86326523181553]
  ALERT is a large-scale benchmark to assess safety based on a novel fine-grained risk taxonomy. It aims to identify vulnerabilities, inform improvements, and enhance the overall safety of the language models.
  arXiv  Detail & Related papers  (2024-04-06T15:01:47Z)
• GSM-Plus: A Comprehensive Benchmark for Evaluating the Robustness of LLMs as Mathematical Problem Solvers [68.77382332826167]
  Large language models (LLMs) have achieved impressive performance across various mathematical reasoning benchmarks. One essential and frequently occurring evidence is that when the math questions are slightly changed, LLMs can behave incorrectly. This motivates us to evaluate the robustness of LLMs' math reasoning capability by testing a wide range of question variations.
  arXiv  Detail & Related papers  (2024-02-29T15:26:14Z)
• Evaluating LLMs' Mathematical and Coding Competency through Ontology-guided Interventions [47.83142414018448]
  We focus on two popular reasoning tasks: arithmetic reasoning and code generation. We introduce (i) a general ontology of perturbations for math and coding questions, (ii) a semi-automatic method to apply these perturbations, and (iii) two datasets. We show a significant performance drop across all the models against perturbed questions.
  arXiv  Detail & Related papers  (2024-01-17T18:13:07Z)
• MathAttack: Attacking Large Language Models Towards Math Solving Ability [29.887497854000276]
  We propose a MathAttack model to attack MWP samples which are closer to the essence of security in solving math problems. It is essential to preserve the mathematical logic of original MWPs during the attacking. Extensive experiments on our RobustMath and two another math benchmark GSM8K and MultiAirth datasets show that MathAttack could effectively attack the math solving ability of LLMs.
  arXiv  Detail & Related papers  (2023-09-04T16:02:23Z)
• JiuZhang: A Chinese Pre-trained Language Model for Mathematical Problem Understanding [74.12405417718054]
  This paper aims to advance the mathematical intelligence of machines by presenting the first Chinese mathematical pre-trained language model(PLM) Unlike other standard NLP tasks, mathematical texts are difficult to understand, since they involve mathematical terminology, symbols and formulas in the problem statement. We design a novel curriculum pre-training approach for improving the learning of mathematical PLMs, consisting of both basic and advanced courses.
  arXiv  Detail & Related papers  (2022-06-13T17:03:52Z)
• TextHide: Tackling Data Privacy in Language Understanding Tasks [54.11691303032022]
  TextHide mitigates privacy risks without slowing down training or reducing accuracy. It requires all participants to add a simple encryption step to prevent an eavesdropping attacker from recovering private text data. We evaluate TextHide on the GLUE benchmark, and our experiments show that TextHide can effectively defend attacks on shared gradients or representations.
  arXiv  Detail & Related papers  (2020-10-12T22:22:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.