VeriEvol: Scaling Multimodal Mathematical Reasoning via Verifiable Evol-Instruct
Abstract Overview
This paper presents VeriEvol, an iterative framework for multimodal mathematical reasoning that separates two scaling problems: making image-grounded math prompts harder and ensuring answer labels are reliable before they are used for training. The system combines type-aware prompt evolution, which rewrites seed image-question pairs into more challenging prompts, with HTV-Agent, an offline hypothesis-test verifier that tries to refute candidate answers using solver disagreement, programmatic checks, and visual evidence. Verified outputs are materialized into both supervised fine-tuning and GRPO-style reinforcement learning data, so the framework can be used without changing the downstream optimizer. The experiments focus on five visual math benchmarks and analyze both end performance and how performance changes as SFT and RL data volumes increase.
Novelty
The main distinctive contribution is framing multimodal RL data construction as a verifiable pipeline in which prompt evolution and answer verification are decoupled before policy optimization. In particular, the paper emphasizes offline falsification of answers through HTV-Agent, rather than assuming labels are correct or trying to fix reliability only inside the RL loop.
Results
On the five-benchmark suite, scaling evolved SFT data from 10K to 250K samples increases mean accuracy from 35.42 to 54.73. With backbone, SFT initialization, and GRPO recipe fixed, the full RL pipeline improves the average from 55.24 for RL-Origin to 59.12, a cumulative gain of 3.88 points, with reported contributions from both evolved prompts and the verifier. The paper also reports monotonic gains when verified RL data is scaled from 10K to 130K samples, and shows the strongest external comparison on MathVerse-VO.
Key Points
- VeriEvol separates prompt-difficulty scaling from answer-reliability control through route-specific evolution and the HTV-Agent verification pipeline.
- The method shows monotonic scaling trends in both supervised and reinforcement-learning data volume, suggesting that the constructed data remains useful as the pool grows.
- Empirically, the full system outperforms its internal baselines across all five benchmarks in the RL setting and is especially strong on image-reliant evaluation such as MathVerse-VO.