Daily Paper Briefing

Abstract Overview

This paper studies multi-turn refinement of scientific diagrams, arguing that single-pass generation often fails to satisfy an author's evolving visual preferences. A formative user study with 14 participants found that all requested additional revisions after seeing an initial draft, and 86% rated the refined diagrams as more satisfactory. To support systematic evaluation, the authors introduce MTPaperBananaBench, a benchmark of 292 diagrams with 3,518 annotated user requirements, together with a user simulator that reveals unsatisfied requirements as natural-language feedback over multiple turns. Their experiments identify two recurring failure modes in existing systems—quality drift across turns and forgetting of previously satisfied requests—and motivate a new multi-agent refinement system, PaperBanana-Interact.

Novelty

The paper presents what it describes as the first benchmark for multi-turn scientific diagram generation, pairing requirement annotations with a simulator for iterative feedback. It also proposes a multi-agent refinement framework that combines summarized interaction memory with a multi-objective critic that explicitly checks current requests, prior requests, source faithfulness, and presentation quality.

Results

Across benchmark settings, PaperBanana-Interact improves over baseline refiners by 11.9–18.6 points in diagram quality and reduces forgetting by 3.7–6.2 points. When refining PaperBanana outputs, it reaches a quality score of 61.2 with 58.0 requirement satisfaction in the k=1 setting and 54.5 quality with 94.2 requirement satisfaction in the k=3 setting, while maintaining lower forgetting than PaperBanana-DirectRefine and generative-model baselines. Human pairwise evaluation on 150 samples also preferred PaperBanana-Interact outputs over NanoBananaPro and PaperBanana-DirectRefine in 81.3% and 76.7% of cases, respectively.

Key Points

1. MTPaperBananaBench provides 292 scientific diagrams and 3,518 annotated requirements for benchmarking multi-turn refinement with simulated user feedback.
2. Baseline multi-turn systems can satisfy many individual requests but commonly suffer from quality drift and from forgetting edits that were successfully applied earlier.
3. PaperBanana-Interact uses a summarizer plus an internal critique-and-refine loop with a multi-objective critic, leading to better quality retention and lower forgetting across turns.

References

• arXiv: https://arxiv.org/abs/2608.30241v1
• Fugu-MT: https://fugumt.com/fugumt/paper_check/2608.30241v1
• Hugging Face Papers: https://huggingface.co/papers/2608.30241