Simple but Efficient: A Multi-Scenario Nearline Retrieval Framework for Recommendation on Taobao

• URL: http://arxiv.org/abs/2408.00247v3
• Date: Tue, 06 Aug 2024 03:27:02 GMT
• Title: Simple but Efficient: A Multi-Scenario Nearline Retrieval Framework for Recommendation on Taobao
• Abstract summary: We introduce an innovative multi-scenario nearline retrieval framework. It operates by harnessing ranking logs from various scenarios through Flink. Our method has shown substantial improvements-most notably, a 5% uptick in product transactions.
• Score: 21.01879010592532
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In recommendation systems, the matching stage is becoming increasingly critical, serving as the upper limit for the entire recommendation process. Recently, some studies have started to explore the use of multi-scenario information for recommendations, such as model-based and data-based approaches. However, the matching stage faces significant challenges due to the need for ultra-large-scale retrieval and meeting low latency requirements. As a result, the methods applied at this stage (collaborative filtering and two-tower models) are often designed to be lightweight, hindering the full utilization of extensive information. On the other hand, the ranking stage features the most sophisticated models with the strongest scoring capabilities, but due to the limited screen size of mobile devices, most of the ranked results may not gain exposure or be displayed. In this paper, we introduce an innovative multi-scenario nearline retrieval framework. It operates by harnessing ranking logs from various scenarios through Flink, allowing us to incorporate finely ranked results from other scenarios into our matching stage in near real-time. Besides, we propose a streaming scoring module, which selects a crucial subset from the candidate pool. Implemented on the "Guess You Like" (homepage of the Taobao APP), China's premier e-commerce platform, our method has shown substantial improvements-most notably, a 5% uptick in product transactions. Furthermore, the proposed approach is not only model-free but also highly efficient, suggesting it can be quickly implemented in diverse scenarios and demonstrate promising performance.

Related papers

• Before Thinking, Learn to Decide: Proactive Routing for Efficient Visual Reasoning [87.18590662029432]
  We propose PRP, a Proactive Routing Paradigm that enables early decision-making by jointly evaluating the competence of both the draft and target models.<n>Our draft Rating Learning (DRL) equips the draft model with an internal confidence estimator, while Joint Rating Learning (JRL) predicts how well the target model can handle a given query.<n>These ratings enable fine-grained, instance-level textbfProactive Routing and substantially accelerate inference without compromising overall performance.
  arXiv  Detail & Related papers  (2026-06-29T12:30:24Z)
• Climber-Pilot: A Non-Myopic Generative Recommendation Model Towards Better Instruction-Following [19.550149895505683]
  We present Climber-Pilot, a unified generative retrieval framework.<n>We introduce Time-Aware Multi-Item Prediction (TAMIP), a novel training paradigm designed to mitigate inherent myopia in generative retrieval.<n>We also propose Condition-Guided Sparse Attention (CGSA), which incorporates business constraints directly into the generative process via sparse attention.
  arXiv  Detail & Related papers  (2026-02-14T03:46:06Z)
• Beyond Existing Retrievals: Cross-Scenario Incremental Sample Learning Framework [14.269153599045152]
  In this paper, we propose a novel retrieval framework IncRec for cross-scenario incremental sample learning.<n>In particular, we deploy IncRec in the Taobao homepage recommendation, achieving a 1% increase in online transaction count.
  arXiv  Detail & Related papers  (2025-12-06T10:26:18Z)
• Breaking the Likelihood Trap: Consistent Generative Recommendation with Graph-structured Model [8.011282322871958]
  Reranking, as the final stage of recommender systems, demands real-time inference, accuracy, and diversity.<n>We propose Consistent Graph-structured Generative Recommendation (Congrats), a novel generative reranking framework.<n>Congrats has been evaluated on a large-scale video-sharing app, Kuaishou, with over 300 million daily active users.
  arXiv  Detail & Related papers  (2025-10-11T09:21:01Z)
• TBGRecall: A Generative Retrieval Model for E-commerce Recommendation Scenarios [31.114619840038095]
  We propose TBGRecall, a framework to enhance generative retrieval models for e-commerce applications.<n>In terms of training methodology, our pipeline integrates limited historical data pre-training with partial incremental training.<n>Our experiments, conducted on public benchmarks alongside a large-scale industrial dataset from TaoBao, show TBGRecall outperforms the state-of-the-art recommendation methods.
  arXiv  Detail & Related papers  (2025-08-16T08:31:11Z)
• Does Multimodality Improve Recommender Systems as Expected? A Critical Analysis and Future Directions [52.21847626165085]
  Multimodal recommendation systems are increasingly popular for their potential to improve performance by integrating diverse data types.<n>However, the actual benefits of this integration remain unclear, raising questions about when and how it truly enhances recommendations.<n>We propose a structured evaluation framework to systematically assess multimodal recommendations across four dimensions.
  arXiv  Detail & Related papers  (2025-08-07T13:21:00Z)
• Can Prompt Difficulty be Online Predicted for Accelerating RL Finetuning of Reasoning Models? [62.579951798437115]
  This work investigates iterative approximate evaluation for arbitrary prompts.<n>It introduces Model Predictive Prompt Selection (MoPPS), a Bayesian risk-predictive framework.<n>MoPPS reliably predicts prompt difficulty and accelerates training with significantly reduced rollouts.
  arXiv  Detail & Related papers  (2025-07-07T03:20:52Z)
• Joint Modeling in Recommendations: A Survey [46.000357352884926]
  Joint modeling approaches are central to overcoming limitations by integrating diverse tasks, scenarios, modalities, and behaviors in the recommendation process. We define the scope of joint modeling through four distinct dimensions: multi-task, multi-scenario, multi-modal, and multi-behavior modeling. We highlight several promising avenues for future exploration in joint modeling for recommendations and provide a concise conclusion to our findings.
  arXiv  Detail & Related papers  (2025-02-28T16:14:00Z)
• Quam: Adaptive Retrieval through Query Affinity Modelling [15.3583908068962]
  Building relevance models to rank documents based on user information needs is a central task in information retrieval and the NLP community. We propose a unifying view of the nascent area of adaptive retrieval by proposing, Quam. Our proposed approach, Quam improves the recall performance by up to 26% over the standard re-ranking baselines.
  arXiv  Detail & Related papers  (2024-10-26T22:52:12Z)
• Unleashing the Potential of Multi-Channel Fusion in Retrieval for Personalized Recommendations [33.79863762538225]
  A key challenge in Recommender systems (RS) is efficiently processing vast item pools to deliver highly personalized recommendations under strict latency constraints. In this paper, we explore advanced channel fusion strategies by assigning systematically optimized weights to each channel. Our methods enhance both personalization and flexibility, achieving significant performance improvements across multiple datasets and yielding substantial gains in real-world deployments.
  arXiv  Detail & Related papers  (2024-10-21T14:58:38Z)
• On the test-time zero-shot generalization of vision-language models: Do we really need prompt learning? [13.803180972839213]
  We introduce a robust MeanShift for Test-time Augmentation (MTA) MTA surpasses prompt-based methods without requiring this intensive training procedure. We extensively benchmark our method on 15 datasets and demonstrate MTA's superiority and computational efficiency.
  arXiv  Detail & Related papers  (2024-05-03T17:34:02Z)
• F$^3$Loc: Fusion and Filtering for Floorplan Localization [63.28504055661646]
  We propose an efficient data-driven solution to self-localization within a floorplan. Our method does not require retraining per map and location or demand a large database of images of the area of interest.
  arXiv  Detail & Related papers  (2024-03-05T23:32:26Z)
• Mirror Gradient: Towards Robust Multimodal Recommender Systems via Exploring Flat Local Minima [54.06000767038741]
  We analyze multimodal recommender systems from the novel perspective of flat local minima. We propose a concise yet effective gradient strategy called Mirror Gradient (MG) We find that the proposed MG can complement existing robust training methods and be easily extended to diverse advanced recommendation models.
  arXiv  Detail & Related papers  (2024-02-17T12:27:30Z)
• Sample Efficient Preference Alignment in LLMs via Active Exploration [63.84454768573154]
  We take advantage of the fact that one can often choose contexts at which to obtain human feedback to most efficiently identify a good policy. We propose an active exploration algorithm to efficiently select the data and provide theoretical proof that it has a worst-case regret bound. Our method outperforms the baselines with limited samples of human preferences on several language models and four real-world datasets.
  arXiv  Detail & Related papers  (2023-12-01T00:54:02Z)
• Towards All-in-one Pre-training via Maximizing Multi-modal Mutual Information [77.80071279597665]
  We propose an all-in-one single-stage pre-training approach, named Maximizing Multi-modal Mutual Information Pre-training (M3I Pre-training) Our approach achieves better performance than previous pre-training methods on various vision benchmarks, including ImageNet classification, object detection, LVIS long-tailed object detection, and ADE20k semantic segmentation.
  arXiv  Detail & Related papers  (2022-11-17T18:59:49Z)
• Temporal Action Detection with Global Segmentation Mask Learning [134.26292288193298]
  Existing temporal action detection (TAD) methods rely on generating an overwhelmingly large number of proposals per video. We propose a proposal-free Temporal Action detection model with Global mask (TAGS) Our core idea is to learn a global segmentation mask of each action instance jointly at the full video length.
  arXiv  Detail & Related papers  (2022-07-14T00:46:51Z)
• Multitarget Tracking with Transformers [21.81266872964314]
  Multitarget Tracking (MTT) is a problem of tracking the states of an unknown number of objects using noisy measurements. In this paper, we propose a high-performing deep-learning method for MTT based on the Transformer architecture.
  arXiv  Detail & Related papers  (2021-04-01T19:14:55Z)
• Learning Salient Boundary Feature for Anchor-free Temporal Action Localization [81.55295042558409]
  Temporal action localization is an important yet challenging task in video understanding. We propose the first purely anchor-free temporal localization method. Our model includes (i) an end-to-end trainable basic predictor, (ii) a saliency-based refinement module, and (iii) several consistency constraints.
  arXiv  Detail & Related papers  (2021-03-24T12:28:32Z)
• Exploration in two-stage recommender systems [79.50534282841618]
  Two-stage recommender systems are widely adopted in industry due to their scalability and maintainability. A key challenge of this setup is that optimal performance of each stage in isolation does not imply optimal global performance. We propose a method of synchronising the exploration strategies between the ranker and the nominators.
  arXiv  Detail & Related papers  (2020-09-01T16:52:51Z)
• Sample-Rank: Weak Multi-Objective Recommendations Using Rejection Sampling [0.5156484100374059]
  We introduce a method involving multi-goal sampling followed by ranking for user-relevance (Sample-Rank) to nudge recommendations towards multi-objective goals of the marketplace. The proposed method's novelty is that it reduces the MO recommendation problem to sampling from a desired multi-goal distribution then using it to build a production-friendly learning-to-rank model.
  arXiv  Detail & Related papers  (2020-08-24T09:17:18Z)
• Self-supervised Learning for Large-scale Item Recommendations [18.19202958502061]
  Large scale recommender models find most relevant items from huge catalogs. With millions to billions of items in the corpus, users tend to provide feedback for a very small set of them. We propose a multi-task self-supervised learning framework for large-scale item recommendations.
  arXiv  Detail & Related papers  (2020-07-25T06:21:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.