MultiBanana: A Challenging Benchmark for Multi-Reference Text-to-Image Generation

• URL: http://arxiv.org/abs/2511.22989v1
• Date: Fri, 28 Nov 2025 08:49:55 GMT
• Title: MultiBanana: A Challenging Benchmark for Multi-Reference Text-to-Image Generation
• Abstract summary: $textbfMultiBanana$ is designed to assess the edge of model capabilities by widely covering multi-reference-specific problems at scale.<n>Our analysis reveals their superior performances, typical failure modes, and areas for improvement.<n>MultiBanana will be released as an open benchmark to push the boundaries and establish a standardized basis for fair comparison in multi-reference image generation.
• Score: 38.21587139161503
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent text-to-image generation models have acquired the ability of multi-reference generation and editing; the ability to inherit the appearance of subjects from multiple reference images and re-render them under new contexts. However, the existing benchmark datasets often focus on the generation with single or a few reference images, which prevents us from measuring the progress on how model performance advances or pointing out their weaknesses, under different multi-reference conditions. In addition, their task definitions are still vague, typically limited to axes such as "what to edit" or "how many references are given", and therefore fail to capture the intrinsic difficulty of multi-reference settings. To address this gap, we introduce $\textbf{MultiBanana}$, which is carefully designed to assesses the edge of model capabilities by widely covering multi-reference-specific problems at scale: (1) varying the number of references, (2) domain mismatch among references (e.g., photo vs. anime), (3) scale mismatch between reference and target scenes, (4) references containing rare concepts (e.g., a red banana), and (5) multilingual textual references for rendering. Our analysis among a variety of text-to-image models reveals their superior performances, typical failure modes, and areas for improvement. MultiBanana will be released as an open benchmark to push the boundaries and establish a standardized basis for fair comparison in multi-reference image generation. Our data and code are available at https://github.com/matsuolab/multibanana .

Related papers

• StructGen: Disambiguating Multi-Reference Image Generation via Structured Context Modeling [112.49355973930555]
  Multi-reference image generation aims to synthesize images by integrating attributes from multiple reference images under textual instructions.<n>Existing approaches, which rely solely on natural language instruction, often fail to capture complex intentions precisely.<n>We propose StructGen, which employs a structured, dictionary-like format to encode multiple reference images.
  arXiv  Detail & Related papers  (2026-07-17T04:35:36Z)
• Rethinking Position Embedding as a Context Controller for Multi-Reference and Multi-Shot Video Generation [6.953982005915382]
  We introduce PoCo (Position Embedding as a Context Controller), which incorporates position encoding as additional context control beyond semantic retrieval.<n>Building on PoCo, we develop a multi-reference and multi-shot video generation model capable of reliably controlling characters with extremely similar visual traits.
  arXiv  Detail & Related papers  (2026-04-04T13:50:38Z)
• MultiBind: A Benchmark for Attribute Misbinding in Multi-Subject Generation [30.628326435554182]
  We introduce MultiBind, a benchmark built from real multi-person photographs.<n>We show that MultiBind reveals binding failures that conventional reconstruction metrics miss.
  arXiv  Detail & Related papers  (2026-03-23T12:54:36Z)
• M$^{3}$T2IBench: A Large-Scale Multi-Category, Multi-Instance, Multi-Relation Text-to-Image Benchmark [39.51629719911405]
  We introduce M$3$T2IBench, a large-scale, multi-category, multi-instance, multi-relation along with an object-detection-based evaluation metric, $AlignScore$.<n>Our findings reveal that current open-source text-to-image models perform poorly on this challenging benchmark.<n>We propose the Revise-Then-Enforce approach to enhance image-text alignment.
  arXiv  Detail & Related papers  (2025-10-27T05:32:50Z)
• Kaleido: Open-Sourced Multi-Subject Reference Video Generation Model [38.79676648965641]
  We present Kaleido, a subject-to-video(S2V) generation framework, which aims to synthesize subject-consistent videos conditioned on multiple reference images of target subjects.<n>Kaleido significantly outperforms previous methods in consistency, fidelity, and generalization, marking an advance in S2V generation.
  arXiv  Detail & Related papers  (2025-10-21T12:28:14Z)
• Openstory++: A Large-scale Dataset and Benchmark for Instance-aware Open-domain Visual Storytelling [81.69474860607542]
  We present Openstory++, a large-scale dataset combining additional instance-level annotations with both images and text. We also present Cohere-Bench, a pioneering benchmark framework for evaluating the image generation tasks when long multimodal context is provided.
  arXiv  Detail & Related papers  (2024-08-07T11:20:37Z)
• Instruct-Imagen: Image Generation with Multi-modal Instruction [90.04481955523514]
  instruct-imagen is a model that tackles heterogeneous image generation tasks and generalizes across unseen tasks. We introduce *multi-modal instruction* for image generation, a task representation articulating a range of generation intents with precision. Human evaluation on various image generation datasets reveals that instruct-imagen matches or surpasses prior task-specific models in-domain.
  arXiv  Detail & Related papers  (2024-01-03T19:31:58Z)
• Towards Better Multi-modal Keyphrase Generation via Visual Entity Enhancement and Multi-granularity Image Noise Filtering [79.44443231700201]
  Multi-modal keyphrase generation aims to produce a set of keyphrases that represent the core points of the input text-image pair. The input text and image are often not perfectly matched, and thus the image may introduce noise into the model. We propose a novel multi-modal keyphrase generation model, which not only enriches the model input with external knowledge, but also effectively filters image noise.
  arXiv  Detail & Related papers  (2023-09-09T09:41:36Z)
• Re-Imagen: Retrieval-Augmented Text-to-Image Generator [58.60472701831404]
  Retrieval-Augmented Text-to-Image Generator (Re-Imagen) Retrieval-Augmented Text-to-Image Generator (Re-Imagen)
  arXiv  Detail & Related papers  (2022-09-29T00:57:28Z)
• Local and Global GANs with Semantic-Aware Upsampling for Image Generation [201.39323496042527]
  We consider generating images using local context. We propose a class-specific generative network using semantic maps as guidance. Lastly, we propose a novel semantic-aware upsampling method.
  arXiv  Detail & Related papers  (2022-02-28T19:24:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.