Graph Exploration Matters: Improving both individual-level and system-level diversity in WeChat Feed Recommender

• URL: http://arxiv.org/abs/2306.00009v1
• Date: Mon, 29 May 2023 19:25:32 GMT
• Title: Graph Exploration Matters: Improving both individual-level and system-level diversity in WeChat Feed Recommender
• Authors: Shuai Yang, Lixin Zhang, Feng Xia, Leyu Lin
• Abstract summary: Individual-level diversity and system-level diversity are both important for industrial recommender systems. We implement and deploy the combined system in WeChat App's Top Stories used by hundreds of millions of users.
• Score: 21.0013026365164
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: There are roughly three stages in real industrial recommendation systems, candidates generation (retrieval), ranking and reranking. Individual-level diversity and system-level diversity are both important for industrial recommender systems. The former focus on each single user's experience, while the latter focus on the difference among users. Graph-based retrieval strategies are inevitably hijacked by heavy users and popular items, leading to the convergence of candidates for users and the lack of system-level diversity. Meanwhile, in the reranking phase, Determinantal Point Process (DPP) is deployed to increase individual-level diverisity. Heavily relying on the semantic information of items, DPP suffers from clickbait and inaccurate attributes. Besides, most studies only focus on one of the two levels of diversity, and ignore the mutual influence among different stages in real recommender systems. We argue that individual-level diversity and system-level diversity should be viewed as an integrated problem, and we provide an efficient and deployable solution for web-scale recommenders. Generally, we propose to employ the retrieval graph information in diversity-based reranking, by which to weaken the hidden similarity of items exposed to users, and consequently gain more graph explorations to improve the system-level diveristy. Besides, we argue that users' propensity for diversity changes over time in content feed recommendation. Therefore, with the explored graph, we also propose to capture the user's real-time personalized propensity to the diversity. We implement and deploy the combined system in WeChat App's Top Stories used by hundreds of millions of users. Offline simulations and online A/B tests show our solution can effectively improve both user engagement and system revenue.

Related papers

• User-Creator Feature Dynamics in Recommender Systems with Dual Influence [19.506536850645343]
  We show that a recommender system with dual influence is guaranteed to polarize, causing diversity loss in the system. We then investigate, both theoretically and empirically, approaches for mitigating polarization and promoting diversity in recommender systems. Unexpectedly, we find that common diversity-promoting approaches do not work in the presence of dual influence, while relevancy-optimizing methods like top-$k$ recommendation can prevent polarization and improve diversity of the system.
  arXiv  Detail & Related papers  (2024-07-19T07:58:26Z)
• Representation Online Matters: Practical End-to-End Diversification in Search and Recommender Systems [8.296711988456762]
  We introduce end-to-end diversification to improve representation in search results and recommendations. We develop, experiment, and deploy scalable diversification mechanisms on the Pinterest platform. Our approaches significantly improve diversity metrics, with a neutral to a positive impact on utility metrics and improved user satisfaction.
  arXiv  Detail & Related papers  (2023-05-24T19:43:26Z)
• Improving Recommendation System Serendipity Through Lexicase Selection [53.57498970940369]
  We propose a new serendipity metric to measure the presence of echo chambers and homophily in recommendation systems. We then attempt to improve the diversity-preservation qualities of well known recommendation techniques by adopting a parent selection algorithm known as lexicase selection. Our results show that lexicase selection, or a mixture of lexicase selection and ranking, outperforms its purely ranked counterparts in terms of personalization, coverage and our specifically designed serendipity benchmark.
  arXiv  Detail & Related papers  (2023-05-18T15:37:38Z)
• PIE: Personalized Interest Exploration for Large-Scale Recommender Systems [0.0]
  We present a framework for exploration in large-scale recommender systems to address these challenges. Our methodology can be easily integrated into an existing large-scale recommender system with minimal modifications. Our work has been deployed in production on Facebook Watch, a popular video discovery and sharing platform serving billions of users.
  arXiv  Detail & Related papers  (2023-04-13T22:25:09Z)
• Stable Bias: Analyzing Societal Representations in Diffusion Models [72.27121528451528]
  We propose a new method for exploring the social biases in Text-to-Image (TTI) systems. Our approach relies on characterizing the variation in generated images triggered by enumerating gender and ethnicity markers in the prompts. We leverage this method to analyze images generated by 3 popular TTI systems and find that while all of their outputs show correlations with US labor demographics, they also consistently under-represent marginalized identities to different extents.
  arXiv  Detail & Related papers  (2023-03-20T19:32:49Z)
• The Minority Matters: A Diversity-Promoting Collaborative Metric Learning Algorithm [154.47590401735323]
  Collaborative Metric Learning (CML) has recently emerged as a popular method in recommendation systems. This paper focuses on a challenging scenario where a user has multiple categories of interests. We propose a novel method called textitDiversity-Promoting Collaborative Metric Learning (DPCML)
  arXiv  Detail & Related papers  (2022-09-30T08:02:18Z)
• kNN-Embed: Locally Smoothed Embedding Mixtures For Multi-interest Candidate Retrieval [7.681386867564213]
  kNN-Embed represents each user as a smoothed mixture over learned item clusters that represent distinct "interests" of the user. We experimentally compare kNN-Embed to standard ANN candidate retrieval, and show significant improvements in overall recall and improved diversity across three datasets.
  arXiv  Detail & Related papers  (2022-05-12T16:42:24Z)
• Simultaneous Relevance and Diversity: A New Recommendation Inference Approach [81.44167398308979]
  We propose a new approach, which extends the general collaborative filtering (CF) by introducing a new way of CF inference, negative-to-positive. Our approach is applicable to a wide range of recommendation scenarios/use-cases at various sophistication levels. Our analysis and experiments on public datasets and real-world production data show that our approach outperforms existing methods on relevance and diversity simultaneously.
  arXiv  Detail & Related papers  (2020-09-27T22:20:12Z)
• Exploration in two-stage recommender systems [79.50534282841618]
  Two-stage recommender systems are widely adopted in industry due to their scalability and maintainability. A key challenge of this setup is that optimal performance of each stage in isolation does not imply optimal global performance. We propose a method of synchronising the exploration strategies between the ranker and the nominators.
  arXiv  Detail & Related papers  (2020-09-01T16:52:51Z)
• A Robust Reputation-based Group Ranking System and its Resistance to Bribery [8.300507994596416]
  We propose a new reputation-based ranking system, utilizing multipartite ratingworks. We study its resistance to bribery and how to design optimal bribing strategies.
  arXiv  Detail & Related papers  (2020-04-13T22:28:29Z)
• Deep Learning for Person Re-identification: A Survey and Outlook [233.36948173686602]
  Person re-identification (Re-ID) aims at retrieving a person of interest across multiple non-overlapping cameras. By dissecting the involved components in developing a person Re-ID system, we categorize it into the closed-world and open-world settings.
  arXiv  Detail & Related papers  (2020-01-13T12:49:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.