VisualLens: Personalization through Visual History

• URL: http://arxiv.org/abs/2411.16034v1
• Date: Mon, 25 Nov 2024 01:45:42 GMT
• Title: VisualLens: Personalization through Visual History
• Authors: Wang Bill Zhu, Deqing Fu, Kai Sun, Yi Lu, Zhaojiang Lin, Seungwhan Moon, Kanika Narang, Mustafa Canim, Yue Liu, Anuj Kumar, Xin Luna Dong,
• Abstract summary: We propose a novel approach, VisualLens, that extracts, filters, and refines image representations, and leverages these signals for personalization. Our approach paves the way for personalized recommendations in scenarios where traditional methods fail.
• Score: 32.938501645752126
• License:
• Abstract: We hypothesize that a user's visual history with images reflecting their daily life, offers valuable insights into their interests and preferences, and can be leveraged for personalization. Among the many challenges to achieve this goal, the foremost is the diversity and noises in the visual history, containing images not necessarily related to a recommendation task, not necessarily reflecting the user's interest, or even not necessarily preference-relevant. Existing recommendation systems either rely on task-specific user interaction logs, such as online shopping history for shopping recommendations, or focus on text signals. We propose a novel approach, VisualLens, that extracts, filters, and refines image representations, and leverages these signals for personalization. We created two new benchmarks with task-agnostic visual histories, and show that our method improves over state-of-the-art recommendations by 5-10% on Hit@3, and improves over GPT-4o by 2-5%. Our approach paves the way for personalized recommendations in scenarios where traditional methods fail.

Related papers

• Interactive Visualization Recommendation with Hier-SUCB [52.11209329270573]
  We propose an interactive personalized visualization recommendation (PVisRec) system that learns on user feedback from previous interactions. For more interactive and accurate recommendations, we propose Hier-SUCB, a contextual semi-bandit in the PVisRec setting.
  arXiv  Detail & Related papers  (2025-02-05T17:14:45Z)
• Personalized Fashion Recommendation with Image Attributes and Aesthetics Assessment [15.423307815155534]
  We aim to provide more accurate personalized fashion recommendations by converting available information, especially images, into two graphs attribute. Compared with previous methods that separate image and text as two components, the proposed method combines image and text information to create a richer attributes graph. Preliminary experiments on the IQON3000 dataset have shown that the proposed method achieves competitive accuracy compared with baselines.
  arXiv  Detail & Related papers  (2025-01-06T15:31:10Z)
• Personalized Graph-Based Retrieval for Large Language Models [51.7278897841697]
  We propose a framework that leverages user-centric knowledge graphs to enrich personalization. By directly integrating structured user knowledge into the retrieval process and augmenting prompts with user-relevant context, PGraph enhances contextual understanding and output quality. We also introduce the Personalized Graph-based Benchmark for Text Generation, designed to evaluate personalized text generation tasks in real-world settings where user history is sparse or unavailable.
  arXiv  Detail & Related papers  (2025-01-04T01:46:49Z)
• Stellar: Systematic Evaluation of Human-Centric Personalized Text-to-Image Methods [52.806258774051216]
  We focus on text-to-image systems that input a single image of an individual and ground the generation process along with text describing the desired visual context. We introduce a standardized dataset (Stellar) that contains personalized prompts coupled with images of individuals that is an order of magnitude larger than existing relevant datasets and where rich semantic ground-truth annotations are readily available. We derive a simple yet efficient, personalized text-to-image baseline that does not require test-time fine-tuning for each subject and which sets quantitatively and in human trials a new SoTA.
  arXiv  Detail & Related papers  (2023-12-11T04:47:39Z)
• Impression-Aware Recommender Systems [53.48892326556546]
  We present a systematic literature review on recommender systems using impressions. We define a theoretical framework to delimit recommender systems using impressions and a novel paradigm for personalized recommendations, called impression-aware recommender systems.
  arXiv  Detail & Related papers  (2023-08-15T16:16:02Z)
• DOR: A Novel Dual-Observation-Based Approach for News Recommendation Systems [2.7648976108201815]
  We propose a novel method to address the problem of news recommendation. Our approach is based on the idea of dual observation. By considering both the content of the news and the user's perspective, our approach is able to provide more personalised and accurate recommendations.
  arXiv  Detail & Related papers  (2023-02-02T22:16:53Z)
• Can you recommend content to creatives instead of final consumers? A RecSys based on user's preferred visual styles [69.69160476215895]
  This report is an extension of the paper "Learning Users' Preferred Visual Styles in an Image Marketplace", presented at ACM RecSys '22. We design a RecSys that learns visual styles preferences to the semantics of the projects users work on.
  arXiv  Detail & Related papers  (2022-08-23T12:11:28Z)
• FaIRCoP: Facial Image Retrieval using Contrastive Personalization [43.293482565385055]
  Retrieving facial images from attributes plays a vital role in various systems such as face recognition and suspect identification. Existing methods do so by comparing specific characteristics from the user's mental image against the suggested images. We propose a method that uses the user's feedback to label images as either similar or dissimilar to the target image.
  arXiv  Detail & Related papers  (2022-05-28T09:52:09Z)
• Personalized Visualization Recommendation [40.838444709402694]
  We introduce the problem of personalized visualization recommendation and present a generic learning framework for solving it. In particular, we focus on recommending visualizations personalized for each individual user based on their past visualization interactions. We release our user-centric visualization corpus consisting of 17.4k users exploring 94k datasets with 2.3 million attributes and 32k user-generated visualizations.
  arXiv  Detail & Related papers  (2021-02-12T04:06:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.