Do Multi-Agents Solve Better Than Single? Evaluating Agentic Frameworks for Diagram-Grounded Geometry Problem Solving and Reasoning

• URL: http://arxiv.org/abs/2512.16698v1
• Date: Thu, 18 Dec 2025 16:00:47 GMT
• Title: Do Multi-Agents Solve Better Than Single? Evaluating Agentic Frameworks for Diagram-Grounded Geometry Problem Solving and Reasoning
• Abstract summary: We compare single-agent and multi-agent pipelines on four visual math benchmarks.<n>For open-source models, multi-agent consistently improves performance.<n>The closed-source Gemini-2.0-Flash generally performs better in single-agent mode.
• Score: 2.0334567793400184
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Diagram-grounded geometry problem solving is a critical benchmark for multimodal large language models (MLLMs), yet the benefits of multi-agent design over single-agent remain unclear. We systematically compare single-agent and multi-agent pipelines on four visual math benchmarks: Geometry3K, MathVerse, OlympiadBench, and We-Math. For open-source models, multi-agent consistently improves performance. For example, Qwen-2.5-VL (7B) gains +6.8 points and Qwen-2.5-VL (32B) gains +3.3 on Geometry3K, and both Qwen-2.5-VL variants see further gains on OlympiadBench and We-Math. In contrast, the closed-source Gemini-2.0-Flash generally performs better in single-agent mode on classic benchmarks, while multi-agent yields only modest improvements on the newer We-Math dataset. These findings show that multi-agent pipelines provide clear benefits for open-source models and can assist strong proprietary systems on newer, less familiar benchmarks, but agentic decomposition is not universally optimal. All code, data, and reasoning files are available at https://github.com/faiyazabdullah/Interpreter-Solver

Related papers

• RestoreBench: Can AI Agents Restore Power Flow Convergence? [2.4326395001229053]
  Diagnosing and resolving non-convergent power flow cases is a promising yet largely unexplored application.<n>We introduce a benchmark that evaluates these capabilities across multiple LLMs and three architectures.<n>The evaluation covers two power grids and 46 cases per grid, each requiring one or more corrective actions to restore convergence.
  arXiv  Detail & Related papers  (2026-08-31T21:15:19Z)
• ExecuGraph: A Multi-Agent, Execution-Grounded Framework for Reliable Backend Code Synthesis with Large Language Models [2.3004655342211078]
  Large Language Models generate plausible backend code, but a single-pass paradigm provides no guarantee of correctness or runtime reliability.<n>We present ExecuGraph, a multi-agent framework that places execution-based validation at the center of backend code.<n>A per-agent ablation, retry-budget sweep, error-class taxonomy, and test-source audit are reported.
  arXiv  Detail & Related papers  (2026-06-18T05:58:17Z)
• Multi-Agent Computer Use [72.79887808312706]
  We argue that we should move towards evaluating and building multi-agent computer use (MACU) systems.<n>We propose a general multi-agent setup in which a manager model decomposes computer use tasks as a directed acyclic graph (DAG)<n>At each iteration, the manager dispatches parallel CUA subagents to carry out nodes on the ready frontier of the DAG.
  arXiv  Detail & Related papers  (2026-06-01T01:29:36Z)
• Graph-of-Agents: A Graph-based Framework for Multi-Agent LLM Collaboration [57.23877089929136]
  Graph-of-Agents (GoA) is a new graph-based framework for modeling multi-agent LLM communication.<n>GoA achieves superior performance using only 3 selected agents, outperforming recent multi-agent LLM baselines.
  arXiv  Detail & Related papers  (2026-04-18T21:13:03Z)
• ISO-Bench: Can Coding Agents Optimize Real-World Inference Workloads? [0.8749675983608171]
  We introduce ISO-Bench, a benchmark for coding agents to test their capabilities on real-world inference tasks.<n>We curated 54 tasks from merged pull requests with measurable performance improvements.
  arXiv  Detail & Related papers  (2026-02-23T08:37:53Z)
• AgentArk: Distilling Multi-Agent Intelligence into a Single LLM Agent [57.10083973844841]
  AgentArk is a novel framework to distill multi-agent dynamics into the weights of a single model.<n>We investigate three hierarchical distillation strategies across various models, tasks, scaling, and scenarios.<n>By shifting the burden of computation from inference to training, the distilled models preserve the efficiency of one agent while exhibiting strong reasoning and self-correction performance of multiple agents.
  arXiv  Detail & Related papers  (2026-02-03T19:18:28Z)
• GenAgent: Scaling Text-to-Image Generation via Agentic Multimodal Reasoning [54.42973725693]
  We introduce GenAgent, unifying visual understanding and generation through an agentic multimodal model.<n>GenAgent significantly boosts base generator(FLUX.1-dev) performance on GenEval++ and WISE.<n>Our framework demonstrates three key properties: 1) cross-tool generalization to generators with varying capabilities, 2) test-time scaling with consistent improvements across interaction rounds, and 3) task-adaptive reasoning that automatically adjusts to different tasks.
  arXiv  Detail & Related papers  (2026-01-26T14:49:04Z)
• AgentGit: A Version Control Framework for Reliable and Scalable LLM-Powered Multi-Agent Systems [7.408263799616532]
  We present AgentGit, a framework that brings Git-like rollback and branching to multi-agent systems (MAS)<n>We show that AgentGit significantly reduces redundant, runtime and token usage, and supports parallel exploration across multiple branches.<n>This work offers a practical path to more robust MAS design and enables error recovery, safe exploration, computation, and A/B testing in collaborative AI systems.
  arXiv  Detail & Related papers  (2025-11-01T17:11:31Z)
• TUMIX: Multi-Agent Test-Time Scaling with Tool-Use Mixture [60.945393748584316]
  We propose an ensemble framework that runs multiple agents in parallel, each employing distinct tool-use strategies and answer paths.<n>TUmix achieves significant gains over state-of-the-art tool-augmented and test-time scaling methods.<n>We find that agent diversity and quality are crucial and can be enhanced by using LLMs to auto-optimize agent designs.
  arXiv  Detail & Related papers  (2025-09-30T19:19:56Z)
• Graph of Agents: Principled Long Context Modeling by Emergent Multi-Agent Collaboration [9.151759069858924]
  We introduce a principled framework that formalizes the model-agnostic long context modeling problem as a compression problem.<n>We propose Graph of Agents (GoA), which dynamically constructs an input-dependent collaboration structure that maximizes this objective.<n>GoA surpasses the 128K context window Llama 3.1 8B on LongBench, showing a dramatic increase in effective context length.
  arXiv  Detail & Related papers  (2025-09-26T04:15:40Z)
• CodeAgents: A Token-Efficient Framework for Codified Multi-Agent Reasoning in LLMs [16.234259194402163]
  We introduce CodeAgents, a prompting framework that codifies multi-agent reasoning and enables structured, token-efficient planning in multi-agent systems.<n>Results show consistent improvements in planning performance, with absolute gains of 3-36 percentage points over natural language prompting baselines.
  arXiv  Detail & Related papers  (2025-07-04T02:20:19Z)
• S*: Test Time Scaling for Code Generation [55.11863577956177]
  We propose S*, the first hybrid test-time scaling framework for code generation.<n>S* substantially improves the coverage and selection accuracy of generated code.
  arXiv  Detail & Related papers  (2025-02-20T09:18:53Z)
• GraphTeam: Facilitating Large Language Model-based Graph Analysis via Multi-Agent Collaboration [31.07238612043854]
  GraphTeam consists of five LLM-based agents from three modules, and the agents with different specialities can collaborate to address complex problems.<n>Experiments on six graph analysis benchmarks demonstrate that GraphTeam achieves state-of-the-art performance with an average 25.85% improvement over the best baseline in terms of accuracy.
  arXiv  Detail & Related papers  (2024-10-23T17:02:59Z)
• Agent-FLAN: Designing Data and Methods of Effective Agent Tuning for Large Language Models [56.00992369295851]
  Open-sourced Large Language Models (LLMs) have achieved great success in various NLP tasks, however, they are still far inferior to API-based models when acting as agents. This paper delivers three key observations: (1) the current agent training corpus is entangled with both formats following and agent reasoning, which significantly shifts from the distribution of its pre-training data; (2) LLMs exhibit different learning speeds on the capabilities required by agent tasks; and (3) current approaches have side-effects when improving agent abilities by introducing hallucinations. We propose Agent-FLAN to effectively Fine-tune LANguage models for Agents.
  arXiv  Detail & Related papers  (2024-03-19T16:26:10Z)
• MAgIC: Investigation of Large Language Model Powered Multi-Agent in Cognition, Adaptability, Rationality and Collaboration [98.18244218156492]
  Large Language Models (LLMs) have significantly advanced natural language processing.<n>As their applications expand into multi-agent environments, there arises a need for a comprehensive evaluation framework.<n>This work introduces a novel competition-based benchmark framework to assess LLMs within multi-agent settings.
  arXiv  Detail & Related papers  (2023-11-14T21:46:27Z)
• MPLP++: Fast, Parallel Dual Block-Coordinate Ascent for Dense Graphical Models [96.1052289276254]
  This work introduces a new MAP-solver, based on the popular Dual Block-Coordinate Ascent principle. Surprisingly, by making a small change to the low-performing solver, we derive the new solver MPLP++ that significantly outperforms all existing solvers by a large margin.
  arXiv  Detail & Related papers  (2020-04-16T16:20:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.