PosterHarness: Turning Scientific Poster Generation into an Auditable Instruction-Following Benchmark

• URL: http://arxiv.org/abs/2607.03006v1
• Date: Fri, 03 Jul 2026 06:39:56 GMT
• Title: PosterHarness: Turning Scientific Poster Generation into an Auditable Instruction-Following Benchmark
• Abstract summary: Text-rich image models can now design poster-scale layouts, but we lack ways to measure whether they honor scientific communication contracts.<n>We present POSTERHARNESS, an auditable harness reframing poster generation as measurable instruction-following tasks.
• Score: 7.957250777212916
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Text-rich image models can now design poster-scale layouts, but we lack ways to measure whether they honor scientific communication contracts: legible labels, prescribed aspect ratios, and -- above all -- abstaining from fabricated scientific figures. We present POSTERHARNESS, an auditable harness reframing poster generation as measurable instruction-following tasks, with a pilot benchmark and failure taxonomy. POSTERHARNESS uses a placeholder-first contract to separate two jobs models otherwise conflate. The model performs visual-summary design: typography, reading path, color, and background -- but never draws data-bearing figures. Every figure region must be an empty labeled placeholder; a deterministic compositor inserts real source-paper figures at detected coordinates. This makes properties measurable: placeholder count and ID accuracy, blankness, aspect-ratio compliance, abstention from synthesized graphics, public-text hygiene, and source-figure provenance -- with failures logged as explicit rejections, not hidden in plausible-looking output. We instantiate the harness on 12 papers (6 HEP, 6 AI/ML-adjacent) and report three findings. (i) A counterfactual probe shows the placeholder contract drives VLM-counted synthesized figures from 34 to 0 across three papers. (ii) A failure taxonomy identifies blocking contracts: placeholder geometry, placeholder QA, template critic, and public text. (iii) Comparison with Paper2Poster shows a trade-off: PosterHarness yields higher-resolution artifacts, lower white-canvas fraction, and stronger VLM visual preference; the deterministic baseline retains slightly more PosterQuiz-style information and runs faster. We report this as regime characterization, not a superiority claim. All artifacts, prompts, manifests, and audit scripts are released as a reusable evaluation component.

Related papers

• From Transient Prompts to Persistent Control: Scientific Poster Generation via Recursive Semantic-Geometric Contracts [67.34566880931716]
  PosterVisor is a control framework that shifts poster generation from transient prompts to persistent control.<n>An Orchestrator grounds rubrics in the paper and visual assets, compiling them into a Semantic-Geometric Contract.<n>Recursive Contract Enforcement (RCE) dynamically triggers checks across stages as evidence emerges.
  arXiv  Detail & Related papers  (2026-09-15T15:32:07Z)
• HALLMARK: Diagnosing Three Failure Modes in LLM Citation Verifiers [30.02579334273979]
  GPTZero found 53 papers with hallucinated citations among NeurIPS 2025's accepted set.<n>No shared benchmark compares verifiers and gives detailed failure diagnostics.<n>Hallucination benchmark: 2,526 Bib entries spanning 14 types, three difficulty tiers, six diagnostic sub-tests per entry.
  arXiv  Detail & Related papers  (2026-07-20T13:21:27Z)
• Automated High-Precision Extraction and Forensic Verification of Data-Bearing Vector Figures [45.39546965981896]
  The quantitative record of science and engineering is increasingly carried by figures rather than text or tables.<n>When a figure is stored as graphics, its data are not approximated by the picture but encoded in it.<n>We turn this into three contributions, one per shortcoming of hand digitization.
  arXiv  Detail & Related papers  (2026-06-30T08:43:32Z)
• Reading $\neq$ Seeing: Diagnosing and Closing the Typography Gap in Vision-Language Models [16.13083848552738]
  Vision-Language Models achieve near-perfect accuracy at reading text in images, yet prove largely typography-blind.<n>We systematically investigate this gap by evaluating font family, size, style, and color recognition across 26 fonts, four scripts, and three difficulty levels.<n>We release our evaluation framework, data, and fine-tuning recipe to support progress in closing the typographic gap in vision-language understanding.
  arXiv  Detail & Related papers  (2026-03-09T15:31:47Z)
• PoSh: Using Scene Graphs To Guide LLMs-as-a-Judge For Detailed Image Descriptions [55.95282725491425]
  PoSh is a metric for detailed image description that uses scene graphs as structured rubrics to guide LLMs-as-a-Judge.<n>PoSh is replicable, interpretable and a better proxy for human raters than existing metrics.<n>We show that PoSh achieves stronger correlations with the human judgments in DOCENT than the best open-weight alternatives.
  arXiv  Detail & Related papers  (2025-10-21T20:30:20Z)
• Losing the Plot: How VLM responses degrade on imperfect charts [11.071294641688496]
  Vision language models (VLMs) show strong results on chart understanding, yet existing benchmarks assume clean figures and fact based queries.<n>We evaluate ChatGPT 4o, Claude Sonnet 4, and Gemini 2.5 Pro, finding sharp performance drops under corruption or occlusion.<n>A key innovation is prompt reverse inconsistency, where models contradict themselves when asked to confirm versus deny the same statement.
  arXiv  Detail & Related papers  (2025-09-22T21:12:20Z)
• Paper2Poster: Towards Multimodal Poster Automation from Scientific Papers [34.9495497884296]
  Poster generation is a crucial yet challenging task in scientific communication.<n>We introduce the first benchmark and metric suite for poster generation.<n>PosterAgent is a top-down, visual-in-the-loop multi-agent pipeline.
  arXiv  Detail & Related papers  (2025-05-27T17:58:49Z)
• Uncertainty-Aware Label Refinement on Hypergraphs for Personalized Federated Facial Expression Recognition [58.98052764581606]
  We develop a novel uncertainty-Aware label refineMent on hYpergraphs (AMY) method.<n>For local training, each local model consists of a backbone, an uncertainty estimation (UE) block, and an expression classification (EC) block.<n>A personalized uncertainty estimator is then introduced to estimate reliable uncertainty weights of samples in the local client.
  arXiv  Detail & Related papers  (2025-01-03T13:59:21Z)
• Noisy-Correspondence Learning for Text-to-Image Person Re-identification [50.07634676709067]
  We propose a novel Robust Dual Embedding method (RDE) to learn robust visual-semantic associations even with noisy correspondences. Our method achieves state-of-the-art results both with and without synthetic noisy correspondences on three datasets.
  arXiv  Detail & Related papers  (2023-08-19T05:34:13Z)
• Towards General Visual-Linguistic Face Forgery Detection [95.73987327101143]
  Deepfakes are realistic face manipulations that can pose serious threats to security, privacy, and trust. Existing methods mostly treat this task as binary classification, which uses digital labels or mask signals to train the detection model. We propose a novel paradigm named Visual-Linguistic Face Forgery Detection(VLFFD), which uses fine-grained sentence-level prompts as the annotation.
  arXiv  Detail & Related papers  (2023-07-31T10:22:33Z)
• FactGraph: Evaluating Factuality in Summarization with Semantic Graph Representations [114.94628499698096]
  We propose FactGraph, a method that decomposes the document and the summary into structured meaning representations (MRs) MRs describe core semantic concepts and their relations, aggregating the main content in both document and summary in a canonical form, and reducing data sparsity. Experiments on different benchmarks for evaluating factuality show that FactGraph outperforms previous approaches by up to 15%.
  arXiv  Detail & Related papers  (2022-04-13T16:45:33Z)
• Weakly-Supervised Salient Object Detection via Scribble Annotations [54.40518383782725]
  We propose a weakly-supervised salient object detection model to learn saliency from scribble labels. We present a new metric, termed saliency structure measure, to measure the structure alignment of the predicted saliency maps. Our method not only outperforms existing weakly-supervised/unsupervised methods, but also is on par with several fully-supervised state-of-the-art models.
  arXiv  Detail & Related papers  (2020-03-17T12:59:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.