PaperBanana-Interact: Scientific Diagram Refinement with Multi-Turn Human Feedback

• URL: http://arxiv.org/abs/2608.30241v1
• Date: Mon, 31 Aug 2026 04:52:32 GMT
• Title: PaperBanana-Interact: Scientific Diagram Refinement with Multi-Turn Human Feedback
• Abstract summary: We present a benchmark for multi-turn diagram generation containing 292 images annotated with 3,518 user requirements.<n>PaperBananaBench identifies unsatisfied requirements and converts k of them into natural language feedback.<n>We introduce PaperBanana-Interact, a multi-agent system that refines diagrams via an internal critique-and-refine loop.
• Score: 101.62446136707372
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent efforts have aimed to automate scientific diagram generation from paper content (Lin et al., 2026; Zhu et al., 2026a). However, fully satisfying an author's visual and communicative preferences in a single turn is challenging: in our formative user study (N = 14), all participants requested further revisions after viewing an initial draft, and 86% of them rated the refined diagrams as more satisfactory. Despite the clear demand, the multi-turn workflow remains largely underexplored. To bridge this gap, we present MTPaperBananaBench, a benchmark for multi-turn diagram generation containing 292 images annotated with 3,518 user requirements. To reduce expensive human studies and enable scalable benchmarking, we construct a user simulator that, at each turn, identifies unsatisfied requirements and converts k of them into natural language feedback. Evaluating both requirement satisfaction and overall diagram quality reveals two key failure modes shared across baseline multiturn systems: (1) quality drift, where diagram quality progressively declines over turns, and (2) forgetting, where previously implemented features are lost in subsequent turns. To address these issues, we introduce PaperBanana-Interact, a multi-agent system that refines diagrams via an internal critique-and-refine loop. PaperBanana-Interact consistently improves rather than degrades diagram quality across turns, outperforming baselines by 11.9-18.6 points in quality score and reducing forgetting by 3.7-6.2 points.

Related papers

• Multi-Agent Debate and Visual Information Extraction for SeePhys Pro: A 1st-Place Technical Report from ICML 2026 AI4Math Track 3 Challenge [26.80223000446421]
  This report presents our approach to the Challenge Track3: SeePhys Pro at the 3rd AI for Math Workshop.<n>The task is to answer college-level physics questions whose statement and figure may be given partly or entirely as an image.<n>We address the task with a two-stage framework: a visual information extraction stage that re-expresses figure content as solver-readable text, and a reasoning stage that orchestrates three heterogeneous solvers.
  arXiv  Detail & Related papers  (2026-07-24T03:44:30Z)
• HABIT: Chrono-Synergia Robust Progressive Learning Framework for Composed Image Retrieval [32.85064790572752]
  Composed Image Retrieval (CIR) is a flexible image retrieval paradigm that enables users to accurately locate the target image.<n>This paper proposes a cHrono-synergiA roBust progressIve learning framework for composed image reTrieval (HABIT)<n>Experiments conducted on two standard CIR datasets demonstrate that HABIT significantly outperforms most methods under various noise ratios.
  arXiv  Detail & Related papers  (2026-04-20T10:02:18Z)
• NTIRE 2026 The 3rd Restore Any Image Model (RAIM) Challenge: Professional Image Quality Assessment (Track 1) [115.35873876159131]
  We present an overview of the NTIRE 2026 challenge on the 3rd Restore Any Image Model in the Wild, specifically focusing on Track 1.<n>In total, the challenge attracted nearly 200 registrations and over 2,500 submissions.<n>The top-performing methods significantly advanced the state of the art in professional IQA.
  arXiv  Detail & Related papers  (2026-04-14T09:44:35Z)
• EduIllustrate: Towards Scalable Automated Generation Of Multimodal Educational Content [19.131221541276332]
  We present EduIllustrate, a benchmark for evaluating large language models.<n>The benchmark comprises 230 problems spanning five subjects and three grade levels.<n>We show that Gemini 3.0 Pro Preview leads at 87.8%, while Kimi-K2.5 achieves the best cost-efficiency.
  arXiv  Detail & Related papers  (2026-04-06T08:58:31Z)
• Iterative Refinement Improves Compositional Image Generation [47.116050084875106]
  Text-to-image (T2I) models struggle with complex prompts that require simultaneously handling multiple objects, relations, and attributes.<n>We propose an iterative test-time strategy in which a T2I model progressively refines its generations across multiple steps.<n>Our approach is simple, requires no external tools or priors, and can be flexibly applied to a wide range of image generators and vision-language models.
  arXiv  Detail & Related papers  (2026-01-21T18:59:40Z)
• COMPOSE: Hypergraph Cover Optimization for Multi-view 3D Human Pose Estimation [58.47973015036709]
  3D pose estimation from sparse multi-views is a critical task for action recognition, sports analysis, and human-robot interaction.<n>We propose COMPOSE, a novel framework that formulates multi-view pose correspondence matching as a hypergraph problem.<n> COMPOSE achieves improvements of up to 23% in average precision over previous optimization-based methods and up to 11% over self-supervised end-to-end learned methods.
  arXiv  Detail & Related papers  (2026-01-14T18:50:17Z)
• PRISMM-Bench: A Benchmark of Peer-Review Grounded Multimodal Inconsistencies [16.537126902822127]
  We introduce PRISMM-Bench, the first benchmark grounded in real reviewer-flagged inconsistencies in scientific papers.<n>We design three tasks, namely inconsistency identification, remedy and pair matching, which assess a model's capacity to detect, correct, and reason over inconsistencies.<n>We benchmark 21 leading LMMs, including large openweight models (GLM-4.5V 106B, InternVL3 78B) and proprietary models (Gemini 2.5 Pro, GPT-5 with high reasoning)
  arXiv  Detail & Related papers  (2025-10-18T13:46:26Z)
• NEVIS'22: A Stream of 100 Tasks Sampled from 30 Years of Computer Vision Research [96.53307645791179]
  We introduce the Never-Ending VIsual-classification Stream (NEVIS'22), a benchmark consisting of a stream of over 100 visual classification tasks. Despite being limited to classification, the resulting stream has a rich diversity of tasks from OCR, to texture analysis, scene recognition, and so forth. Overall, NEVIS'22 poses an unprecedented challenge for current sequential learning approaches due to the scale and diversity of tasks.
  arXiv  Detail & Related papers  (2022-11-15T18:57:46Z)
• Summarization with Graphical Elements [55.5913491389047]
  We propose a new task: summarization with graphical elements. We collect a high quality human labeled dataset to support research into the task.
  arXiv  Detail & Related papers  (2022-04-15T17:16:41Z)
• Learning multiview 3D point cloud registration [74.39499501822682]
  We present a novel, end-to-end learnable, multiview 3D point cloud registration algorithm. Our approach outperforms the state-of-the-art by a significant margin, while being end-to-end trainable and computationally less costly.
  arXiv  Detail & Related papers  (2020-01-15T03:42:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.