MODRL-TA:A Multi-Objective Deep Reinforcement Learning Framework for Traffic Allocation in E-Commerce Search

• URL: http://arxiv.org/abs/2407.15476v1
• Date: Mon, 22 Jul 2024 08:40:27 GMT
• Title: MODRL-TA:A Multi-Objective Deep Reinforcement Learning Framework for Traffic Allocation in E-Commerce Search
• Authors: Peng Cheng, Huimu Wang, Jinyuan Zhao, Yihao Wang, Enqiang Xu, Yu Zhao, Zhuojian Xiao, Songlin Wang, Guoyu Tang, Lin Liu, Sulong Xu,
• Abstract summary: This paper propose a multi-objective deep reinforcement learning framework consisting of multi-objective Q-learning (MOQ), a decision fusion algorithm (DFM) based on the cross-entropy method(CEM) and a progressive data augmentation system(PDA) Experiments on real-world online e-commerce systems demonstrate the significant improvements of MODRL-TA.
• Score: 13.893431289065997
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Traffic allocation is a process of redistributing natural traffic to products by adjusting their positions in the post-search phase, aimed at effectively fostering merchant growth, precisely meeting customer demands, and ensuring the maximization of interests across various parties within e-commerce platforms. Existing methods based on learning to rank neglect the long-term value of traffic allocation, whereas approaches of reinforcement learning suffer from balancing multiple objectives and the difficulties of cold starts within realworld data environments. To address the aforementioned issues, this paper propose a multi-objective deep reinforcement learning framework consisting of multi-objective Q-learning (MOQ), a decision fusion algorithm (DFM) based on the cross-entropy method(CEM), and a progressive data augmentation system(PDA). Specifically. MOQ constructs ensemble RL models, each dedicated to an objective, such as click-through rate, conversion rate, etc. These models individually determine the position of items as actions, aiming to estimate the long-term value of multiple objectives from an individual perspective. Then we employ DFM to dynamically adjust weights among objectives to maximize long-term value, addressing temporal dynamics in objective preferences in e-commerce scenarios. Initially, PDA trained MOQ with simulated data from offline logs. As experiments progressed, it strategically integrated real user interaction data, ultimately replacing the simulated dataset to alleviate distributional shifts and the cold start problem. Experimental results on real-world online e-commerce systems demonstrate the significant improvements of MODRL-TA, and we have successfully deployed MODRL-TA on an e-commerce search platform.

Related papers

• Multi-Stream Cellular Test-Time Adaptation of Real-Time Models Evolving in Dynamic Environments [53.79708667153109]
  Smart objects, notably autonomous vehicles, face challenges in critical local computations due to limited resources. We propose a novel Multi-Stream Cellular Test-Time Adaptation setup where models adapt on the fly to a dynamic environment divided into cells. We validate our methodology in the context of autonomous vehicles navigating across cells defined based on location and weather conditions.
  arXiv  Detail & Related papers  (2024-04-27T15:00:57Z)
• LiMAML: Personalization of Deep Recommender Models via Meta Learning [13.69036196446634]
  We introduce an innovative meta-learning solution tailored to the personalization of models for individual members and other entities. We leverage the Model-Agnostic Meta Learning (MAML) algorithm to adapt per-task sub-networks using recent user interaction data. Our approach has enabled the deployment of a range of highly personalized AI models across diverse LinkedIn applications.
  arXiv  Detail & Related papers  (2024-02-23T22:06:36Z)
• Communication-Efficient Multimodal Federated Learning: Joint Modality and Client Selection [14.261582708240407]
  Multimodal Federated learning (FL) aims to enrich model training in FL settings where clients are collecting measurements across multiple modalities. Key challenges to multimodal FL remain unaddressed, particularly in heterogeneous network settings. We propose mmFedMC, a new FL methodology that can tackle the above-mentioned challenges in multimodal settings.
  arXiv  Detail & Related papers  (2024-01-30T02:16:19Z)
• MOTO: Offline Pre-training to Online Fine-tuning for Model-based Robot Learning [52.101643259906915]
  We study the problem of offline pre-training and online fine-tuning for reinforcement learning from high-dimensional observations. Existing model-based offline RL methods are not suitable for offline-to-online fine-tuning in high-dimensional domains. We propose an on-policy model-based method that can efficiently reuse prior data through model-based value expansion and policy regularization.
  arXiv  Detail & Related papers  (2024-01-06T21:04:31Z)
• Data-Efficient Task Generalization via Probabilistic Model-based Meta Reinforcement Learning [58.575939354953526]
  PACOH-RL is a novel model-based Meta-Reinforcement Learning (Meta-RL) algorithm designed to efficiently adapt control policies to changing dynamics. Existing Meta-RL methods require abundant meta-learning data, limiting their applicability in settings such as robotics. Our experiment results demonstrate that PACOH-RL outperforms model-based RL and model-based Meta-RL baselines in adapting to new dynamic conditions.
  arXiv  Detail & Related papers  (2023-11-13T18:51:57Z)
• A Meta-learning based Stacked Regression Approach for Customer Lifetime Value Prediction [3.6002910014361857]
  Customer Lifetime Value (CLV) is the total monetary value of transactions/purchases made by a customer with the business over an intended period of time. CLV finds application in a number of distinct business domains such as Banking, Insurance, Online-entertainment, Gaming, and E-Commerce. We propose a system which is able to qualify both as effective, and comprehensive yet simple and interpretable.
  arXiv  Detail & Related papers  (2023-08-07T14:22:02Z)
• FedDM: Iterative Distribution Matching for Communication-Efficient Federated Learning [87.08902493524556]
  Federated learning(FL) has recently attracted increasing attention from academia and industry. We propose FedDM to build the global training objective from multiple local surrogate functions. In detail, we construct synthetic sets of data on each client to locally match the loss landscape from original data.
  arXiv  Detail & Related papers  (2022-07-20T04:55:18Z)
• Entity-Graph Enhanced Cross-Modal Pretraining for Instance-level Product Retrieval [152.3504607706575]
  This research aims to conduct weakly-supervised multi-modal instance-level product retrieval for fine-grained product categories. We first contribute the Product1M datasets, and define two real practical instance-level retrieval tasks. We exploit to train a more effective cross-modal model which is adaptively capable of incorporating key concept information from the multi-modal data.
  arXiv  Detail & Related papers  (2022-06-17T15:40:45Z)
• Learning to Continuously Optimize Wireless Resource in a Dynamic Environment: A Bilevel Optimization Perspective [52.497514255040514]
  This work develops a new approach that enables data-driven methods to continuously learn and optimize resource allocation strategies in a dynamic environment. We propose to build the notion of continual learning into wireless system design, so that the learning model can incrementally adapt to the new episodes. Our design is based on a novel bilevel optimization formulation which ensures certain fairness" across different data samples.
  arXiv  Detail & Related papers  (2021-05-03T07:23:39Z)
• Relation-aware Meta-learning for Market Segment Demand Prediction with Limited Records [40.33535461064516]
  We propose a novel algorithm, RMLDP, to incorporate a multi-pattern fusion network (MPFN) with a meta-learning paradigm. We conduct extensive experiments on two large-scale industrial datasets. The results justify that our RMLDP outperforms a set of state-of-the-art baselines.
  arXiv  Detail & Related papers  (2020-08-01T06:02:16Z)
• MoTiAC: Multi-Objective Actor-Critics for Real-Time Bidding [47.555870679348416]
  We propose a Multi-ecTive Actor-Critics algorithm named MoTiAC for the problem of bidding optimization with various goals. Unlike previous RL models, the proposed MoTiAC can simultaneously fulfill multi-objective tasks in complicated bidding environments.
  arXiv  Detail & Related papers  (2020-02-18T07:16:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.