MRSE: An Efficient Multi-modality Retrieval System for Large Scale E-commerce

• URL: http://arxiv.org/abs/2408.14968v1
• Date: Tue, 27 Aug 2024 11:21:19 GMT
• Title: MRSE: An Efficient Multi-modality Retrieval System for Large Scale E-commerce
• Authors: Hao Jiang, Haoxiang Zhang, Qingshan Hou, Chaofeng Chen, Weisi Lin, Jingchang Zhang, Annan Wang,
• Abstract summary: Current Embedding-based Retrieval Systems embed queries and items into a shared low-dimensional space. We propose MRSE, a Multi-modality Retrieval System that integrates text, item images, and user preferences. MRSE achieves an 18.9% improvement in offline relevance and a 3.7% gain in online core metrics compared to Shopee's state-of-the-art uni-modality system.
• Score: 42.3177388371158
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Providing high-quality item recall for text queries is crucial in large-scale e-commerce search systems. Current Embedding-based Retrieval Systems (ERS) embed queries and items into a shared low-dimensional space, but uni-modality ERS rely too heavily on textual features, making them unreliable in complex contexts. While multi-modality ERS incorporate various data sources, they often overlook individual preferences for different modalities, leading to suboptimal results. To address these issues, we propose MRSE, a Multi-modality Retrieval System that integrates text, item images, and user preferences through lightweight mixture-of-expert (LMoE) modules to better align features across and within modalities. MRSE also builds user profiles at a multi-modality level and introduces a novel hybrid loss function that enhances consistency and robustness using hard negative sampling. Experiments on a large-scale dataset from Shopee and online A/B testing show that MRSE achieves an 18.9% improvement in offline relevance and a 3.7% gain in online core metrics compared to Shopee's state-of-the-art uni-modality system.

Related papers

• CUE-M: Contextual Understanding and Enhanced Search with Multimodal Large Language Model [9.451422869974557]
  This paper introduces Con Understanding and Enhanced Search with MLLM (CUE-M), a novel multimodaltextual search pipeline. Evaluations on a multimodal Q&A dataset and a public safety benchmark demonstrate that CUE-M outperforms baselines in accuracy, knowledge integration, and safety.
  arXiv  Detail & Related papers  (2024-11-19T07:16:48Z)
• Multimodality Helps Few-Shot 3D Point Cloud Semantic Segmentation [61.91492500828508]
  Few-shot 3D point cloud segmentation (FS-PCS) aims at generalizing models to segment novel categories with minimal support samples. We introduce a cost-free multimodal FS-PCS setup, utilizing textual labels and the potentially available 2D image modality. We propose a simple yet effective Test-time Adaptive Cross-modal Seg (TACC) technique to mitigate training bias.
  arXiv  Detail & Related papers  (2024-10-29T19:28:41Z)
• ATFLRec: A Multimodal Recommender System with Audio-Text Fusion and Low-Rank Adaptation via Instruction-Tuned Large Language Model [0.0]
  This study examines the integration of multimodal data text and audio into large language models (LLMs) Traditional text and audio recommenders encounter limitations such as the cold-start problem. Low-Rank Adaptation (LoRA) is introduced, which enhances efficiency without compromising performance.
  arXiv  Detail & Related papers  (2024-09-13T05:33:09Z)
• Smart Multi-Modal Search: Contextual Sparse and Dense Embedding Integration in Adobe Express [3.8973445113342433]
  Building a scalable multi-modal search system requires fine-tuning several components. We address considerations such as embedding model selection, the roles of embeddings in matching and ranking, and the balance between dense and sparse embeddings.
  arXiv  Detail & Related papers  (2024-08-26T23:52:27Z)
• An Interactive Multi-modal Query Answering System with Retrieval-Augmented Large Language Models [21.892975397847316]
  We present an interactive Multi-modal Query Answering (MQA) system, empowered by our newly developed multi-modal retrieval framework and navigation graph index. One notable aspect of MQA is its utilization of contrastive learning to assess the significance of different modalities. The system achieves efficient retrieval through our advanced navigation graph index, refined using computational pruning techniques.
  arXiv  Detail & Related papers  (2024-07-05T02:01:49Z)
• A Thorough Performance Benchmarking on Lightweight Embedding-based Recommender Systems [67.52782366565658]
  State-of-the-art recommender systems (RSs) depend on categorical features, which ecoded by embedding vectors, resulting in excessively large embedding tables. Despite the prosperity of lightweight embedding-based RSs, a wide diversity is seen in evaluation protocols. This study investigates various LERS' performance, efficiency, and cross-task transferability via a thorough benchmarking process.
  arXiv  Detail & Related papers  (2024-06-25T07:45:00Z)
• LLM-ESR: Large Language Models Enhancement for Long-tailed Sequential Recommendation [58.04939553630209]
  In real-world systems, most users interact with only a handful of items, while the majority of items are seldom consumed. These two issues, known as the long-tail user and long-tail item challenges, often pose difficulties for existing Sequential Recommendation systems. We propose the Large Language Models Enhancement framework for Sequential Recommendation (LLM-ESR) to address these challenges.
  arXiv  Detail & Related papers  (2024-05-31T07:24:42Z)
• Multimodal Recommender Systems: A Survey [50.23505070348051]
  Multimodal Recommender System (MRS) has attracted much attention from both academia and industry recently. In this paper, we will give a comprehensive survey of the MRS models, mainly from technical views. To access more details of the surveyed papers, such as implementation code, we open source a repository.
  arXiv  Detail & Related papers  (2023-02-08T05:12:54Z)
• ItemSage: Learning Product Embeddings for Shopping Recommendations at Pinterest [60.841761065439414]
  At Pinterest, we build a single set of product embeddings called ItemSage to provide relevant recommendations in all shopping use cases. This approach has led to significant improvements in engagement and conversion metrics, while reducing both infrastructure and maintenance cost.
  arXiv  Detail & Related papers  (2022-05-24T02:28:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.