ImageGem: In-the-wild Generative Image Interaction Dataset for Generative Model Personalization

• URL: http://arxiv.org/abs/2510.18433v1
• Date: Tue, 21 Oct 2025 09:08:01 GMT
• Title: ImageGem: In-the-wild Generative Image Interaction Dataset for Generative Model Personalization
• Authors: Yuanhe Guo, Linxi Xie, Zhuoran Chen, Kangrui Yu, Ryan Po, Guandao Yang, Gordon Wetztein, Hongyi Wen,
• Abstract summary: ImageGem is a dataset for studying generative models that understand fine-grained individual preferences.<n>Our dataset features real-world interaction data from 57K users, who collectively have built 242K customized LoRAs, written 3M text prompts, and created 5M generated images.
• Score: 11.7261367003714
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce ImageGem, a dataset for studying generative models that understand fine-grained individual preferences. We posit that a key challenge hindering the development of such a generative model is the lack of in-the-wild and fine-grained user preference annotations. Our dataset features real-world interaction data from 57K users, who collectively have built 242K customized LoRAs, written 3M text prompts, and created 5M generated images. With user preference annotations from our dataset, we were able to train better preference alignment models. In addition, leveraging individual user preference, we investigated the performance of retrieval models and a vision-language model on personalized image retrieval and generative model recommendation. Finally, we propose an end-to-end framework for editing customized diffusion models in a latent weight space to align with individual user preferences. Our results demonstrate that the ImageGem dataset enables, for the first time, a new paradigm for generative model personalization.

Related papers

• One Adapts to Any: Meta Reward Modeling for Personalized LLM Alignment [55.86333374784959]
  We argue that addressing these constraints requires a paradigm shift from fitting data to learn user preferences to learn the process of preference adaptation.<n>We propose Meta Reward Modeling (MRM), which reformulates personalized reward modeling as a meta-learning problem.<n>We show that MRM enhances few-shot personalization, improves user robustness, and consistently outperforms baselines.
  arXiv  Detail & Related papers  (2026-01-26T17:55:52Z)
• Personalized Image Editing in Text-to-Image Diffusion Models via Collaborative Direct Preference Optimization [11.306247975771013]
  Collaborative Preference Optimization (C-DPO) is a novel method that aligns image edits with user-specific preferences.<n>Our approach encodes each user as a node in a dynamic preference graph and learns embeddings via a lightweight graph neural network.<n>Our method consistently outperforms baselines in generating edits that are aligned with user preferences.
  arXiv  Detail & Related papers  (2025-11-06T18:59:54Z)
• Anyprefer: An Agentic Framework for Preference Data Synthesis [62.3856754548222]
  We propose Anyprefer, a framework designed to synthesize high-quality preference data for aligning the target model.<n> external tools are introduced to assist the judge model in accurately rewarding the target model's responses.<n>The synthesized data is compiled into a new preference dataset, Anyprefer-V1, consisting of 58K high-quality preference pairs.
  arXiv  Detail & Related papers  (2025-04-27T15:21:59Z)
• Personalized Preference Fine-tuning of Diffusion Models [75.22218338096316]
  We introduce PPD, a multi-reward optimization objective that aligns diffusion models with personalized preferences.<n>With PPD, a diffusion model learns the individual preferences of a population of users in a few-shot way.<n>Our approach achieves an average win rate of 76% over Stable Cascade, generating images that more accurately reflect specific user preferences.
  arXiv  Detail & Related papers  (2025-01-11T22:38:41Z)
• Multi-subject Open-set Personalization in Video Generation [110.02124633005516]
  We present Video Alchemist $-$ a video model with built-in multi-subject, open-set personalization capabilities.<n>Our model is built on a new Diffusion Transformer module that fuses each conditional reference image and its corresponding subject-level text prompt.<n>Our method significantly outperforms existing personalization methods in both quantitative and qualitative evaluations.
  arXiv  Detail & Related papers  (2025-01-10T18:59:54Z)
• You Only Submit One Image to Find the Most Suitable Generative Model [48.67303250592189]
  We propose a novel setting called Generative Model Identification (GMI)<n>GMI aims to enable the user to identify the most appropriate generative model(s) for the user's requirements efficiently.
  arXiv  Detail & Related papers  (2024-12-16T14:46:57Z)
• Preference Adaptive and Sequential Text-to-Image Generation [24.787970969428976]
  We create a novel dataset of sequential preferences, which we leverage, together with large-scale open-source (non-sequential) datasets.<n>We construct user-preference and user-choice models using an EM strategy and identify varying user preference types.<n>We then leverage a large multimodal language model (LMM) and a value-based RL approach to suggest an adaptive and diverse slate of prompt expansions to the user.<n>Our Preference Adaptive and Sequential Text-to-image Agent (PASTA) extends T2I models with adaptive multi-turn capabilities, fostering collaborative co-creation and addressing uncertainty or underspecification
  arXiv  Detail & Related papers  (2024-12-10T01:47:40Z)
• MIA-DPO: Multi-Image Augmented Direct Preference Optimization For Large Vision-Language Models [85.30735602813093]
  Multi-Image Augmented Direct Preference Optimization (MIA-DPO) is a visual preference alignment approach that effectively handles multi-image inputs. MIA-DPO mitigates the scarcity of diverse multi-image training data by extending single-image data with unrelated images arranged in grid collages or pic-in-pic formats.
  arXiv  Detail & Related papers  (2024-10-23T07:56:48Z)
• ViPer: Visual Personalization of Generative Models via Individual Preference Learning [11.909247529297678]
  We propose to personalize the image generation process by capturing the generic preferences of the user in a one-time process. Based on these comments, we infer a user's structured liked and disliked visual attributes. These attributes are used to guide a text-to-image model toward producing images that are tuned towards the individual user's visual preference.
  arXiv  Detail & Related papers  (2024-07-24T15:42:34Z)
• JeDi: Joint-Image Diffusion Models for Finetuning-Free Personalized Text-to-Image Generation [49.997839600988875]
  Existing personalization methods rely on finetuning a text-to-image foundation model on a user's custom dataset. We propose Joint-Image Diffusion (jedi), an effective technique for learning a finetuning-free personalization model. Our model achieves state-of-the-art generation quality, both quantitatively and qualitatively, significantly outperforming both the prior finetuning-based and finetuning-free personalization baselines.
  arXiv  Detail & Related papers  (2024-07-08T17:59:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.