RAGAR: Retrieval Augment Personalized Image Generation Guided by Recommendation

• URL: http://arxiv.org/abs/2505.01657v1
• Date: Sat, 03 May 2025 02:20:30 GMT
• Title: RAGAR: Retrieval Augment Personalized Image Generation Guided by Recommendation
• Authors: Run Ling, Wenji Wang, Yuting Liu, Guibing Guo, Linying Jiang, Xingwei Wang,
• Abstract summary: We propose Retrieval Augment Personalized Image GenerAtion guided by Recommendation (RAGAR)<n>Our approach uses a retrieval mechanism to assign different weights to historical items according to their similarities to the reference item.<n> RAGAR achieves significant improvements in both personalization and semantic metrics compared to five baselines.
• Score: 9.31199434211423
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Personalized image generation is crucial for improving the user experience, as it renders reference images into preferred ones according to user visual preferences. Although effective, existing methods face two main issues. First, existing methods treat all items in the user historical sequence equally when extracting user preferences, overlooking the varying semantic similarities between historical items and the reference item. Disproportionately high weights for low-similarity items distort users' visual preferences for the reference item. Second, existing methods heavily rely on consistency between generated and reference images to optimize the generation, which leads to underfitting user preferences and hinders personalization. To address these issues, we propose Retrieval Augment Personalized Image GenerAtion guided by Recommendation (RAGAR). Our approach uses a retrieval mechanism to assign different weights to historical items according to their similarities to the reference item, thereby extracting more refined users' visual preferences for the reference item. Then we introduce a novel rank task based on the multi-modal ranking model to optimize the personalization of the generated images instead of forcing depend on consistency. Extensive experiments and human evaluations on three real-world datasets demonstrate that RAGAR achieves significant improvements in both personalization and semantic metrics compared to five baselines.

Related papers

• ConsRec: Denoising Sequential Recommendation through User-Consistent Preference Modeling [33.281526528724335]
  We propose the User-Consistent Preference-based Sequential Recommendation System (ConsRec)<n>ConsRec captures stable user preferences and filters noisy items from interaction histories.<n>Results show ConsRec achieves a 13% improvement over baseline recommendation models.
  arXiv  Detail & Related papers  (2025-05-28T08:55:13Z)
• Optimizing Multi-Round Enhanced Training in Diffusion Models for Improved Preference Understanding [29.191627597682597]
  We present a framework incorporating human-in-the-loop feedback, leveraging a well-trained reward model aligned with user preferences.<n>Our approach consistently surpasses competing models in user satisfaction, especially in multi-turn dialogue scenarios.
  arXiv  Detail & Related papers  (2025-04-25T09:35:02Z)
• DRC: Enhancing Personalized Image Generation via Disentangled Representation Composition [69.10628479553709]
  We introduce DRC, a novel personalized image generation framework that enhances Large Multimodal Models (LMMs)<n> DRC explicitly extracts user style preferences and semantic intentions from history images and the reference image, respectively.<n>It involves two critical learning stages: 1) Disentanglement learning, which employs a dual-tower disentangler to explicitly separate style and semantic features, optimized via a reconstruction-driven paradigm with difficulty-aware importance sampling; and 2) Personalized modeling, which applies semantic-preserving augmentations to effectively adapt the disentangled representations for robust personalized generation.
  arXiv  Detail & Related papers  (2025-04-24T08:10:10Z)
• Interactive Visualization Recommendation with Hier-SUCB [52.11209329270573]
  We propose an interactive personalized visualization recommendation (PVisRec) system that learns on user feedback from previous interactions.<n>For more interactive and accurate recommendations, we propose Hier-SUCB, a contextual semi-bandit in the PVisRec setting.
  arXiv  Detail & Related papers  (2025-02-05T17:14:45Z)
• Personalized Fashion Recommendation with Image Attributes and Aesthetics Assessment [15.423307815155534]
  We aim to provide more accurate personalized fashion recommendations by converting available information, especially images, into two graphs attribute.<n>Compared with previous methods that separate image and text as two components, the proposed method combines image and text information to create a richer attributes graph.<n>Preliminary experiments on the IQON3000 dataset have shown that the proposed method achieves competitive accuracy compared with baselines.
  arXiv  Detail & Related papers  (2025-01-06T15:31:10Z)
• VisualLens: Personalization through Visual History [32.938501645752126]
  We propose a novel approach, VisualLens, that extracts, filters, and refines image representations, and leverages these signals for personalization. Our approach paves the way for personalized recommendations in scenarios where traditional methods fail.
  arXiv  Detail & Related papers  (2024-11-25T01:45:42Z)
• ViPer: Visual Personalization of Generative Models via Individual Preference Learning [11.909247529297678]
  We propose to personalize the image generation process by capturing the generic preferences of the user in a one-time process. Based on these comments, we infer a user's structured liked and disliked visual attributes. These attributes are used to guide a text-to-image model toward producing images that are tuned towards the individual user's visual preference.
  arXiv  Detail & Related papers  (2024-07-24T15:42:34Z)
• Ada-Retrieval: An Adaptive Multi-Round Retrieval Paradigm for Sequential Recommendations [50.03560306423678]
  We propose Ada-Retrieval, an adaptive multi-round retrieval paradigm for recommender systems. Ada-Retrieval iteratively refines user representations to better capture potential candidates in the full item space.
  arXiv  Detail & Related papers  (2024-01-12T15:26:40Z)
• Feature Decoupling-Recycling Network for Fast Interactive Segmentation [79.22497777645806]
  Recent interactive segmentation methods iteratively take source image, user guidance and previously predicted mask as the input. We propose the Feature Decoupling-Recycling Network (FDRN), which decouples the modeling components based on their intrinsic discrepancies.
  arXiv  Detail & Related papers  (2023-08-07T12:26:34Z)
• Contextual Similarity Aggregation with Self-attention for Visual Re-ranking [96.55393026011811]
  We propose a visual re-ranking method by contextual similarity aggregation with self-attention. We conduct comprehensive experiments on four benchmark datasets to demonstrate the generality and effectiveness of our proposed visual re-ranking method.
  arXiv  Detail & Related papers  (2021-10-26T06:20:31Z)
• Set2setRank: Collaborative Set to Set Ranking for Implicit Feedback based Recommendation [59.183016033308014]
  In this paper, we explore the unique characteristics of the implicit feedback and propose Set2setRank framework for recommendation. Our proposed framework is model-agnostic and can be easily applied to most recommendation prediction approaches.
  arXiv  Detail & Related papers  (2021-05-16T08:06:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.