Paper2Poster: Towards Multimodal Poster Automation from Scientific Papers

• URL: http://arxiv.org/abs/2505.21497v1
• Date: Tue, 27 May 2025 17:58:49 GMT
• Title: Paper2Poster: Towards Multimodal Poster Automation from Scientific Papers
• Authors: Wei Pang, Kevin Qinghong Lin, Xiangru Jian, Xi He, Philip Torr,
• Abstract summary: Poster generation is a crucial yet challenging task in scientific communication.<n>We introduce the first benchmark and metric suite for poster generation.<n>PosterAgent is a top-down, visual-in-the-loop multi-agent pipeline.
• Score: 11.186078920251754
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Academic poster generation is a crucial yet challenging task in scientific communication, requiring the compression of long-context interleaved documents into a single, visually coherent page. To address this challenge, we introduce the first benchmark and metric suite for poster generation, which pairs recent conference papers with author-designed posters and evaluates outputs on (i)Visual Quality-semantic alignment with human posters, (ii)Textual Coherence-language fluency, (iii)Holistic Assessment-six fine-grained aesthetic and informational criteria scored by a VLM-as-judge, and notably (iv)PaperQuiz-the poster's ability to convey core paper content as measured by VLMs answering generated quizzes. Building on this benchmark, we propose PosterAgent, a top-down, visual-in-the-loop multi-agent pipeline: the (a)Parser distills the paper into a structured asset library; the (b)Planner aligns text-visual pairs into a binary-tree layout that preserves reading order and spatial balance; and the (c)Painter-Commenter loop refines each panel by executing rendering code and using VLM feedback to eliminate overflow and ensure alignment. In our comprehensive evaluation, we find that GPT-4o outputs-though visually appealing at first glance-often exhibit noisy text and poor PaperQuiz scores, and we find that reader engagement is the primary aesthetic bottleneck, as human-designed posters rely largely on visual semantics to convey meaning. Our fully open-source variants (e.g. based on the Qwen-2.5 series) outperform existing 4o-driven multi-agent systems across nearly all metrics, while using 87% fewer tokens. It transforms a 22-page paper into a finalized yet editable .pptx poster - all for just $0.005. These findings chart clear directions for the next generation of fully automated poster-generation models. The code and datasets are available at https://github.com/Paper2Poster/Paper2Poster.

Related papers

• PosterCraft: Rethinking High-Quality Aesthetic Poster Generation in a Unified Framework [26.60241017305203]
  PosterCraft is a unified framework that abandons prior modular pipelines and rigid, predefined layouts.<n>It employs a carefully designed, cascaded workflow to optimize the generation of high-aesthetic posters.<n>PosterCraft significantly outperforms open-source baselines in rendering accuracy, layout coherence, and overall visual appeal.
  arXiv  Detail & Related papers  (2025-06-12T14:28:12Z)
• P2P: Automated Paper-to-Poster Generation and Fine-Grained Benchmark [27.57464219790922]
  We introduce P2P, the first flexible, LLM-based multi-agent framework that generates high-quality, HTML-rendered academic posters.<n>P2P employs three specialized agents-for visual element processing, content generation, and final poster assembly-each integrated with dedicated checker modules.<n>We establish P2PEval, a comprehensive benchmark featuring 121 paper-poster pairs and a dual evaluation methodology.
  arXiv  Detail & Related papers  (2025-05-21T09:06:05Z)
• PosterMaker: Towards High-Quality Product Poster Generation with Accurate Text Rendering [50.76106125697899]
  Product posters, which integrate subject, scene, and text, are crucial promotional tools for attracting customers.<n>Main challenge lies in accurately rendering text, especially for complex writing systems like Chinese, which contains over 10,000 individual characters.<n>We develop TextRenderNet, which achieves a high text rendering accuracy of over 90%.<n>Based on TextRenderNet and SceneGenNet, we present PosterMaker, an end-to-end generation framework.
  arXiv  Detail & Related papers  (2025-04-09T07:13:08Z)
• PosterSum: A Multimodal Benchmark for Scientific Poster Summarization [19.416714365519713]
  PosterSum is a novel benchmark to advance the development of vision-language models.<n>We benchmark state-of-the-art Multimodal Large Language Models (MLLMs) on PosterSum.<n>We propose Segment & Summarize, a hierarchical method that outperforms current MLLMs on automated metrics.
  arXiv  Detail & Related papers  (2025-02-24T18:35:39Z)
• PPTAgent: Generating and Evaluating Presentations Beyond Text-to-Slides [51.88536367177796]
  We propose a two-stage, edit-based approach inspired by human drafts for automatically generating presentations.<n>PWTAgent first analyzes references to extract slide-level functional types and content schemas, then generates editing actions based on selected reference slides.<n>PWTAgent significantly outperforms existing automatic presentation generation methods across all three dimensions.
  arXiv  Detail & Related papers  (2025-01-07T16:53:01Z)
• mPLUG-DocOwl2: High-resolution Compressing for OCR-free Multi-page Document Understanding [103.05835688963947]
  We propose a High-resolution DocCompressor module to compress each high-resolution document image into 324 tokens. DocOwl2 sets a new state-of-the-art across multi-page document understanding benchmarks and reduces first token latency by more than 50%. Compared to single-image MLLMs trained on similar data, our DocOwl2 achieves comparable single-page understanding performance with less than 20% of the visual tokens.
  arXiv  Detail & Related papers  (2024-09-05T11:09:00Z)
• GlyphDraw2: Automatic Generation of Complex Glyph Posters with Diffusion Models and Large Language Models [7.152732507491591]
  We propose an automatic poster generation framework with text rendering capabilities leveraging LLMs.<n>This framework aims to create precise poster text within a detailed contextual background.<n>We introduce a high-resolution font dataset and a poster dataset with resolutions exceeding 1024 pixels.
  arXiv  Detail & Related papers  (2024-07-02T13:17:49Z)
• OmniParser: A Unified Framework for Text Spotting, Key Information Extraction and Table Recognition [79.852642726105]
  We propose a unified paradigm for parsing visually-situated text across diverse scenarios. Specifically, we devise a universal model, called Omni, which can simultaneously handle three typical visually-situated text parsing tasks. In Omni, all tasks share the unified encoder-decoder architecture, the unified objective point-conditioned text generation, and the unified input representation.
  arXiv  Detail & Related papers  (2024-03-28T03:51:14Z)
• Uncovering Prototypical Knowledge for Weakly Open-Vocabulary Semantic Segmentation [59.37587762543934]
  This paper studies the problem of weakly open-vocabulary semantic segmentation (WOVSS) Existing methods suffer from a granularity inconsistency regarding the usage of group tokens. We propose the prototypical guidance network (PGSeg) that incorporates multi-modal regularization.
  arXiv  Detail & Related papers  (2023-10-29T13:18:00Z)
• KOSMOS-2.5: A Multimodal Literate Model [136.96172068766285]
  We present KOSMOS-2.5, a multimodal literate model for machine reading of text-intensive images. KOSMOS-2.5 excels in two distinct yet complementary transcription tasks. We fine-tune KOSMOS-2.5 for document understanding tasks, resulting in a document understanding generalist named KOSMOS-2.5-CHAT.
  arXiv  Detail & Related papers  (2023-09-20T15:50:08Z)
• Text2Poster: Laying out Stylized Texts on Retrieved Images [32.466518932018175]
  Poster generation is a significant task for a wide range of applications, which is often time-consuming and requires lots of manual editing and artistic experience. We propose a novel data-driven framework, called textitText2Poster, to automatically generate visually-effective posters from textual information.
  arXiv  Detail & Related papers  (2023-01-06T04:06:23Z)
• Long Document Summarization with Top-down and Bottom-up Inference [113.29319668246407]
  We propose a principled inference framework to improve summarization models on two aspects. Our framework assumes a hierarchical latent structure of a document where the top-level captures the long range dependency. We demonstrate the effectiveness of the proposed framework on a diverse set of summarization datasets.
  arXiv  Detail & Related papers  (2022-03-15T01:24:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.