MIBE: Multi-subject Interaction Benchmark and Evaluator for Personalized Image Generation

• URL: http://arxiv.org/abs/2607.01383v1
• Date: Wed, 01 Jul 2026 18:44:01 GMT
• Title: MIBE: Multi-subject Interaction Benchmark and Evaluator for Personalized Image Generation
• Abstract summary: We introduce a unified framework comprising a Multi-subject Interaction Benchmark (MIB) and a Multi-subject Interaction Evaluator (MIE)<n>MIB systematically covers diverse relation types and scene complexities through a decoupled data regime.<n>MIE exhibits strong cross-generator generalization on the Gold Set, achieving 0.922 overall pairwise accuracy against human preference.
• Score: 12.891921839026432
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Multi-subject personalized image generation requires the precise rendering of all requested reference identities and their specified interactions based on a guiding prompt. However, state-of-the-art models still struggle with this process, frequently omitting subjects, failing to preserve reference appearances, or misattributing interactions. Furthermore, existing metrics designed primarily for single-subject fidelity cannot reliably capture these errors, suffering severe degradation in ranking separability and failing to align with human preference as the subject count increases. To address this gap, we introduce Multi-subject Interaction Benchmark and Evaluator (MIBE), a unified framework comprising a Multi-subject Interaction Benchmark (MIB) and a Multi-subject Interaction Evaluator (MIE). MIB systematically covers diverse relation types and scene complexities through a decoupled data regime. This consists of a 60K-pair VLM-labeled Silver Set for scalable metric training and a 4K-pair double-blind Human Evaluation Gold Set covering a diverse range of state-of-the-art generators, with the Silver Set reaching 95.1% cross-VLM preference agreement. To demonstrate the utility of this benchmark, we present MIE, a lightweight, reference-conditioned evaluator trained exclusively on the Silver Set with a dual-head ranking and diagnosis objective. MIE exhibits strong cross-generator generalization on the Gold Set, achieving 0.922 overall pairwise accuracy against human preference, including 0.982 on seen generators and 0.884 on unseen generators. By outperforming a broad spectrum of baseline metrics, including CLIP and DINO variants, MIE demonstrates that diagnostic supervision can preserve ranking separability and human alignment where traditional evaluators collapse.

Related papers

• Inter-X++: A Comprehensive Benchmark for Multimodal Human-Human Interaction Analysis [55.78822230833247]
  We present Inter-X++, a comprehensive and large-scale benchmark designed to empower versatile HHI analysis.<n>Inter-X++ provides 11,388 high-fidelity interaction sequences and over 8.1M frames, featuring precise whole-body movements and detailed finger articulations.<n>We propose OpenHHI, a single and unified HHI representation and modeling framework that jointly optimize interaction reconstruction and semantic understanding.
  arXiv  Detail & Related papers  (2026-08-20T17:51:48Z)
• PerceptionRubrics: Calibrating Multimodal Evaluation to Human Perception [75.70273182182397]
  PerceptionRubrics is a rubric-based evaluation framework.<n>It addresses the gap between saturated benchmark scores and real-world brittleness.<n>It pairs 1,038 information-dense images with over 10,000 instance-specific rubrics.
  arXiv  Detail & Related papers  (2026-06-26T17:59:15Z)
• DualEval: Joint Model-Item Calibration for Unified LLM Evaluation [67.99119926012686]
  DualEval is a latent model-item calibration framework that represents models and evaluation items in a shared space.<n>We apply DualEval across four domains: coding, math, miscellaneous domain-knowledge tasks, and generic everyday user queries.
  arXiv  Detail & Related papers  (2026-06-24T22:40:46Z)
• Visual Aesthetic Benchmark: Can Frontier Models Judge Beauty? [59.923111838399144]
  We introduce the Visual Aesthetic Benchmark (VAB), which casts aesthetic evaluation as comparative selection over candidate sets with matched subject matter.<n>VAB contains 400 tasks and 1,195 images across fine art, photography, and illustration, with labels derived from the consensus of 10 independent expert judges per task.<n>We find that the strongest system identifies both the best and the worst image correctly across three random permutations of the candidate order in only 26.5% of tasks, far below the 68.9% achieved by human experts.
  arXiv  Detail & Related papers  (2026-05-12T19:33:28Z)
• Enhancing Value Alignment of LLMs with Multi-agent system and Combinatorial Fusion [5.97790552623697]
  We propose a framework that operationalizes multi-agent fusion alignment.<n>It mitigates conflicts and redundancies across multiple agents, producing responses that better reflect human values.
  arXiv  Detail & Related papers  (2026-03-11T14:57:51Z)
• MSVBench: Towards Human-Level Evaluation of Multi-Shot Video Generation [48.84450712826316]
  MSVBench is the first comprehensive benchmark featuring hierarchical scripts and reference images tailored for Multi-Shot Video generation.<n>We propose a hybrid evaluation framework that synergizes the high-level semantic reasoning of Large Multimodal Models with the fine-grained perceptual rigor of domain-specific expert models.
  arXiv  Detail & Related papers  (2026-02-27T12:26:34Z)
• GenArena: How Can We Achieve Human-Aligned Evaluation for Visual Generation Tasks? [29.804627410258732]
  We introduce a unified evaluation framework that leverages a pairwise comparison paradigm to ensure stable and human-aligned evaluation.<n>Our method boosts evaluation accuracy by over 20% and achieves a Spearman correlation of 0.86 with the authoritative LMArena leaderboard.
  arXiv  Detail & Related papers  (2026-02-05T18:52:48Z)
• Relative Classification Accuracy: A Calibrated Metric for Identity Consistency in Fine-Grained K-pop Face Generation [0.0]
  Denoising Diffusion Probabilistic Models (DDPMs) have achieved remarkable success in high-fidelity image generation.<n>Standard metrics like FID and Inception Score (IS) often fail to detect identity misalignment in such specialized contexts.<n>We investigate Class-Conditional DDPMs for K-pop idol face generation (32x32), a domain characterized by high inter-class similarity.
  arXiv  Detail & Related papers  (2026-01-22T00:58:59Z)
• KG-EDAS: A Meta-Metric Framework for Evaluating Knowledge Graph Completion Models [0.0]
  A major challenge in evaluating Knowledge Graphs (KGs) is comparing their performance across multiple datasets and metrics.<n>We propose KG Evaluation based on Distance from Average Solution (EDAS) to integrate multi-metric, multi-dataset performance into a unified ranking.<n>EDAS offers a global perspective that supports more informed model selection and promotes fairness in cross-dataset evaluation.
  arXiv  Detail & Related papers  (2025-08-21T08:37:35Z)
• A Quality-Guided Mixture of Score-Fusion Experts Framework for Human Recognition [14.242680363313148]
  Whole-body biometric recognition is a challenging task that integrates various biometric modalities.<n>We present textbfQuality-guided textbfMixture of score-fusion textbfExperts (QME)<n>We introduce a novel pseudo-quality loss for quality estimation with a modality-specific Quality Estimator (QE) and a score triplet loss to improve the metric performance.
  arXiv  Detail & Related papers  (2025-07-31T18:00:01Z)
• ArtifactsBench: Bridging the Visual-Interactive Gap in LLM Code Generation Evaluation [51.297873393639456]
  ArtifactsBench is a framework for automated visual code generation evaluation.<n>Our framework renders each generated artifact and captures its dynamic behavior through temporal screenshots.<n>We construct a new benchmark of 1,825 diverse tasks and evaluate over 30 leading Large Language Models.
  arXiv  Detail & Related papers  (2025-07-07T12:53:00Z)
• MMIE: Massive Multimodal Interleaved Comprehension Benchmark for Large Vision-Language Models [71.36392373876505]
  We introduce MMIE, a large-scale benchmark for evaluating interleaved multimodal comprehension and generation in Large Vision-Language Models (LVLMs)<n>MMIE comprises 20K meticulously curated multimodal queries, spanning 3 categories, 12 fields, and 102 subfields, including mathematics, coding, physics, literature, health, and arts.<n>It supports both interleaved inputs and outputs, offering a mix of multiple-choice and open-ended question formats to evaluate diverse competencies.
  arXiv  Detail & Related papers  (2024-10-14T04:15:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.