Aphanta: Diagnosing Task-Aligned Image-Edited Intermediates for Multimodal Reasoning

• URL: http://arxiv.org/abs/2608.26993v1
• Date: Thu, 27 Aug 2026 11:41:15 GMT
• Title: Aphanta: Diagnosing Task-Aligned Image-Edited Intermediates for Multimodal Reasoning
• Abstract summary: Explicit visual intermediates can help multimodal large language models externalize spatial evidence and updated visual states.<n>We introduce textbfAphanta, an automated task-discovery and closed-loop diagnostic framework for the MLLM -> image editor -> MLLM pipeline.<n>Aphanta evaluates three conditions---direct reasoning, reasoning with an editor-generated intermediate, and reasoning with an idealized reference intermediate.
• Score: 40.55876706740581
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Explicit visual intermediates can help multimodal large language models (MLLMs) externalize spatial evidence and updated visual states, but their utility depends on whether an image editor can faithfully realize the required transformation. We introduce \textbf{Aphanta}, an automated task-discovery and closed-loop diagnostic framework for the MLLM -> image editor -> MLLM pipeline. Aphanta evaluates three conditions---direct reasoning, reasoning with an editor-generated intermediate, and reasoning with an idealized reference intermediate---to separate potential visual headroom from the practical utility of current editors. Across 20 candidate tasks and multiple editor--MLLM combinations, we find that utility is strongly task-conditioned. Gains concentrate in visual cue injection, grounding, and counterfactual state realization, whereas intermediates requiring symbol-sensitive construction or structural extrapolation are substantially less reliable. On the selected positive-task subset, our consolidated Qwen pipeline improves the mean task score from 0.343 to 0.445 ($+10.2$ points; $+29.7\%$ relative), while the full study also retains filtered and unsuccessful tasks to expose the boundary. These results position image editing as a specialized visual workspace rather than a universal reasoning mechanism, and establish Aphanta as a reusable protocol for measuring task--representation alignment, editor realization, and downstream pipeline utility.

Related papers

• CanvasAgent: Enabling Complex Image Creation and Editing via Visual Tool Orchestration [11.992882604393003]
  We introduce CanvasCraft, a large-scale multimodal tool-use dataset for complex image creation and editing.<n>We also present CanvasAgent, a toolaugmented multimodal agent that learns to orchestrate heterogeneous visual tools through multi-turn interaction.
  arXiv  Detail & Related papers  (2026-07-06T04:57:18Z)
• IEA: Amateur-Friendly Conversational Image Editing Agent via Three Stages of Multitask Alignment [16.199788035961564]
  We propose IEA, a conversational Image Editing Agent that learns to operate parameterized tools in an explicit, interpretable action space.<n>IEA is trained via a three-stage multitask pipeline: SFT on distilled expert edits, GRPO with rewards for likeness improvement, tool usefulness, and intent summarization, and (3) large-scale synthetic fine-tuning to jointly master image editing, refinement, and user intent summarization.
  arXiv  Detail & Related papers  (2026-06-06T07:11:08Z)
• MM-CondChain: A Programmatically Verified Benchmark for Visually Grounded Deep Compositional Reasoning [37.02901476078596]
  We introduce MM-CondChain, a benchmark for visually grounded deep compositional reasoning.<n> MM-CondChain is organized as a multi-layer reasoning chain, where every layer contains a non-trivial compositional condition.<n>A Planner orchestrates layer-by-layer generation of compositional conditions, while a Verifiable Programmatic Intermediate Representation (VPIR) ensures each layer's condition is mechanically verifiable.
  arXiv  Detail & Related papers  (2026-03-12T17:59:56Z)
• IMAGAgent: Orchestrating Multi-Turn Image Editing via Constraint-Aware Planning and Reflection [40.21337735524356]
  IMAGAgent is a multi-turn image editing agent framework based on a "plan-execute-reflect" closed-loop mechanism.<n>It achieves deep synergy among instruction parsing, tool scheduling, and adaptive correction within a unified pipeline.<n>Experiments on our constructed textbfMTEditBench and the MagicBrush dataset demonstrate that IMAGAgent achieves performance significantly superior to existing methods.
  arXiv  Detail & Related papers  (2026-02-12T02:37:38Z)
• VULCAN: Tool-Augmented Multi Agents for Iterative 3D Object Arrangement [66.13644883379087]
  We tackle three key challenges in 3D object arrangement task using MLLMs.<n>First, to address the weak visual grounding of MLLMs, we introduce an MCP-based API.<n>Second, we augment the MLLM's 3D scene understanding with a suite of specialized visual tools.<n>Third, to manage the iterative, error-prone updates, we propose a collaborative multi-agent framework.
  arXiv  Detail & Related papers  (2025-12-26T19:22:39Z)
• Code-in-the-Loop Forensics: Agentic Tool Use for Image Forgery Detection [59.04089915447622]
  ForenAgent is an interactive IFD framework that enables MLLMs to autonomously generate, execute, and refine Python-based low-level tools around the detection objective.<n>Inspired by human reasoning, we design a dynamic reasoning loop comprising global perception, local focusing, iterative probing, and holistic adjudication.<n>Experiments show that ForenAgent exhibits emergent tool-use competence and reflective reasoning on challenging IFD tasks.
  arXiv  Detail & Related papers  (2025-12-18T08:38:44Z)
• Visual Document Understanding and Question Answering: A Multi-Agent Collaboration Framework with Test-Time Scaling [83.78874399606379]
  We propose MACT, a Multi-Agent Collaboration framework with Test-Time scaling.<n>It comprises four distinct small-scale agents, with clearly defined roles and effective collaboration.<n>It shows superior performance with a smaller parameter scale without sacrificing the ability of general and mathematical tasks.
  arXiv  Detail & Related papers  (2025-08-05T12:52:09Z)
• FOCUS: Unified Vision-Language Modeling for Interactive Editing Driven by Referential Segmentation [55.01077993490845]
  Recent Large Vision Language Models (LVLMs) demonstrate promising capabilities in unifying visual understanding and generative modeling.<n>We introduce FOCUS, a unified LVLM that integrates segmentation-aware perception and controllable object-centric generation within an end-to-end framework.
  arXiv  Detail & Related papers  (2025-06-20T07:46:40Z)
• Towards Text-Image Interleaved Retrieval [49.96332254241075]
  We introduce the text-image interleaved retrieval (TIIR) task, where the query and document are interleaved text-image sequences.<n>We construct a TIIR benchmark based on naturally interleaved wikiHow tutorials, where a specific pipeline is designed to generate interleaved queries.<n>We propose a novel Matryoshka Multimodal Embedder (MME), which compresses the number of visual tokens at different granularity.
  arXiv  Detail & Related papers  (2025-02-18T12:00:47Z)
• Good Visual Guidance Makes A Better Extractor: Hierarchical Visual Prefix for Multimodal Entity and Relation Extraction [88.6585431949086]
  We propose a novel Hierarchical Visual Prefix fusion NeTwork (HVPNeT) for visual-enhanced entity and relation extraction. We regard visual representation as pluggable visual prefix to guide the textual representation for error insensitive forecasting decision. Experiments on three benchmark datasets demonstrate the effectiveness of our method, and achieve state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-05-07T02:10:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.