Residual Multi-Task Learner for Applied Ranking

• URL: http://arxiv.org/abs/2411.09705v1
• Date: Wed, 30 Oct 2024 06:49:45 GMT
• Title: Residual Multi-Task Learner for Applied Ranking
• Authors: Cong Fu, Kun Wang, Jiahua Wu, Yizhou Chen, Guangda Huzhang, Yabo Ni, Anxiang Zeng, Zhiming Zhou,
• Abstract summary: ResFlow is a lightweight multi-task learning framework that enables efficient cross-task information sharing. It is fully deployed in the pre-rank module of Shopee Search.
• Score: 11.774841918446137
• License:
• Abstract: Modern e-commerce platforms rely heavily on modeling diverse user feedback to provide personalized services. Consequently, multi-task learning has become an integral part of their ranking systems. However, existing multi-task learning methods encounter two main challenges: some lack explicit modeling of task relationships, resulting in inferior performance, while others have limited applicability due to being computationally intensive, having scalability issues, or relying on strong assumptions. To address these limitations and better fit our real-world scenario, pre-rank in Shopee Search, we introduce in this paper ResFlow, a lightweight multi-task learning framework that enables efficient cross-task information sharing via residual connections between corresponding layers of task networks. Extensive experiments on datasets from various scenarios and modalities demonstrate its superior performance and adaptability over state-of-the-art methods. The online A/B tests in Shopee Search showcase its practical value in large-scale industrial applications, evidenced by a 1.29% increase in OPU (order-per-user) without additional system latency. ResFlow is now fully deployed in the pre-rank module of Shopee Search. To facilitate efficient online deployment, we propose a novel offline metric Weighted Recall@K, which aligns well with our online metric OPU, addressing the longstanding online-offline metric misalignment issue. Besides, we propose to fuse scores from the multiple tasks additively when ranking items, which outperforms traditional multiplicative fusion. The code is released at https://github.com/BrunoTruthAlliance/ResFlow

Related papers

• Cross-Task Affinity Learning for Multitask Dense Scene Predictions [5.939164722752263]
  Multitask learning (MTL) has become prominent for its ability to predict multiple tasks jointly. We introduce the Cross-Task Affinity Learning (CTAL) module, a lightweight framework that enhances task refinement in multitask networks. Our results demonstrate state-of-the-art MTL performance for both CNN and transformer backbones, using significantly fewer parameters than single-task learning.
  arXiv  Detail & Related papers  (2024-01-20T05:31:47Z)
• Multitask Learning with No Regret: from Improved Confidence Bounds to Active Learning [79.07658065326592]
  Quantifying uncertainty in the estimated tasks is of pivotal importance for many downstream applications, such as online or active learning. We provide novel multitask confidence intervals in the challenging setting when neither the similarity between tasks nor the tasks' features are available to the learner. We propose a novel online learning algorithm that achieves such improved regret without knowing this parameter in advance.
  arXiv  Detail & Related papers  (2023-08-03T13:08:09Z)
• Provable Multi-Task Representation Learning by Two-Layer ReLU Neural Networks [69.38572074372392]
  We present the first results proving that feature learning occurs during training with a nonlinear model on multiple tasks. Our key insight is that multi-task pretraining induces a pseudo-contrastive loss that favors representations that align points that typically have the same label across tasks.
  arXiv  Detail & Related papers  (2023-07-13T16:39:08Z)
• Diffusion Model is an Effective Planner and Data Synthesizer for Multi-Task Reinforcement Learning [101.66860222415512]
  Multi-Task Diffusion Model (textscMTDiff) is a diffusion-based method that incorporates Transformer backbones and prompt learning for generative planning and data synthesis. For generative planning, we find textscMTDiff outperforms state-of-the-art algorithms across 50 tasks on Meta-World and 8 maps on Maze2D.
  arXiv  Detail & Related papers  (2023-05-29T05:20:38Z)
• Multi-Task Fusion via Reinforcement Learning for Long-Term User Satisfaction in Recommender Systems [3.4394890850129007]
  We propose a Batch Reinforcement Learning based Multi-Task Fusion framework (BatchRL-MTF) We learn an optimal recommendation policy from the fixed batch data offline for long-term user satisfaction. With a comprehensive investigation on user behaviors, we model the user satisfaction reward with subtles from two aspects of user stickiness and user activeness.
  arXiv  Detail & Related papers  (2022-08-09T06:35:05Z)
• All Birds with One Stone: Multi-task Text Classification for Efficient Inference with One Forward Pass [34.85886030306857]
  In web content classification, multiple classification tasks are predicted from same input text such as a web article. Existing multitask transformer models need to conduct N forward passes for N tasks with O(N) cost. We propose a scalable method that can achieve stronger performance with close to O(1) computation cost via only one forward pass.
  arXiv  Detail & Related papers  (2022-05-22T05:16:03Z)
• Provable and Efficient Continual Representation Learning [40.78975699391065]
  In continual learning (CL), the goal is to design models that can learn a sequence of tasks without catastrophic forgetting. We study the problem of continual representation learning where we learn an evolving representation as new tasks arrive. We show that CL benefits if the initial tasks have large sample size and high "representation diversity"
  arXiv  Detail & Related papers  (2022-03-03T21:23:08Z)
• Decoupled and Memory-Reinforced Networks: Towards Effective Feature Learning for One-Step Person Search [65.51181219410763]
  One-step methods have been developed to handle pedestrian detection and identification sub-tasks using a single network. There are two major challenges in the current one-step approaches. We propose a decoupled and memory-reinforced network (DMRNet) to overcome these problems.
  arXiv  Detail & Related papers  (2021-02-22T06:19:45Z)
• Reparameterizing Convolutions for Incremental Multi-Task Learning without Task Interference [75.95287293847697]
  Two common challenges in developing multi-task models are often overlooked in literature. First, enabling the model to be inherently incremental, continuously incorporating information from new tasks without forgetting the previously learned ones (incremental learning) Second, eliminating adverse interactions amongst tasks, which has been shown to significantly degrade the single-task performance in a multi-task setup (task interference)
  arXiv  Detail & Related papers  (2020-07-24T14:44:46Z)
• MTI-Net: Multi-Scale Task Interaction Networks for Multi-Task Learning [82.62433731378455]
  We show that tasks with high affinity at a certain scale are not guaranteed to retain this behaviour at other scales. We propose a novel architecture, namely MTI-Net, that builds upon this finding.
  arXiv  Detail & Related papers  (2020-01-19T21:02:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.