LLMDistill4Ads: Using Cross-Encoders to Distill from LLM Signals for Advertiser Keyphrase Recommendations at eBay

• URL: http://arxiv.org/abs/2508.03628v1
• Date: Tue, 05 Aug 2025 16:47:17 GMT
• Title: LLMDistill4Ads: Using Cross-Encoders to Distill from LLM Signals for Advertiser Keyphrase Recommendations at eBay
• Authors: Soumik Dey, Benjamin Braun, Naveen Ravipati, Hansi Wu, Binbin Li,
• Abstract summary: This study introduces a novel two-step LLM distillation process from a LLM-judge used to debias our Embedding Based Retrieval (EBR) model from click-data.<n>We distill from an LLM teacher via a cross-encoder assistant into a bi-encoder student using a multi-task training approach, ultimately employing the student bi-encoder to retrieve relevant advertiser keyphrases.
• Score: 1.4555205338313157
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Sellers at eBay are recommended keyphrases to bid on to enhance the performance of their advertising campaigns. The relevance of these keyphrases is crucial in avoiding the overcrowding of search systems with irrelevant items and maintaining a positive seller perception. It is essential that keyphrase recommendations align with both seller and Search judgments regarding auctions. Due to the difficulty in procuring negative human judgment at scale, employing LLM-as-a-judge to mimic seller judgment has been established as the norm in several studies. This study introduces a novel two-step LLM distillation process from a LLM-judge used to debias our Embedding Based Retrieval (EBR) model from the various biases that exist in click-data. We distill from an LLM teacher via a cross-encoder assistant into a bi-encoder student using a multi-task training approach, ultimately employing the student bi-encoder to retrieve relevant advertiser keyphrases. We show that integrating a knowledge distillation process from LLMs in a multi-task training setup enhances bi-encoder performance in retrieving relevant advertiser keyphrases at eBay.

Related papers

• Learning from Synthetic Labs: Language Models as Auction Participants [12.007281866970485]
  This paper introduces a novel synthetic data-generating process to help facilitate the study and design of auctions.<n>We find that simulated AI agents (large language models) agree with the experimental literature in auctions across a variety of classic formats.
  arXiv  Detail & Related papers  (2025-07-12T00:00:30Z)
• Multi-objective Aligned Bidword Generation Model for E-commerce Search Advertising [16.8420671443003]
  Retrieval systems primarily address the challenge of matching user queries with the most relevant advertisements.<n>We propose a Multi-objective aligned Bidword Generation Model (MoBGM), which is composed of a discriminator, generator, and preference alignment module.<n>Our proposed algorithm significantly outperforms the state of the art in offline and online experiments.
  arXiv  Detail & Related papers  (2025-06-04T10:57:18Z)
• To Judge or not to Judge: Using LLM Judgements for Advertiser Keyphrase Relevance at eBay [1.7058804466282262]
  E-commerce sellers are recommended keyphrases based on their inventory to increase buyer engagement (clicks/sales)<n> relevance of advertiser keyphrases plays an important role in preventing the inundation of search systems with numerous irrelevant items.<n>This study discusses the practicalities of using human judgment via a case study at eBay Advertising.
  arXiv  Detail & Related papers  (2025-05-07T08:03:25Z)
• Middleman Bias in Advertising: Aligning Relevance of Keyphrase Recommendations with Search [4.275764895529604]
  We describe the shortcomings of training relevance filter models on biased click/sales signals.<n>We re-conceptualize advertiser keyphrase relevance as interaction between two dynamical systems.<n>We discuss the bias of search relevance systems and the need to align advertiser keyphrases with search relevance signals.
  arXiv  Detail & Related papers  (2025-01-31T19:28:26Z)
• Best Practices for Distilling Large Language Models into BERT for Web Search Ranking [14.550458167328497]
  Large Language Models (LLMs) can generate a ranked list of potential documents. We transfer the ranking expertise of LLMs to a more compact model like BERT, using a ranking loss to enable the deployment of less resource-intensive models. Our model has been successfully integrated into a commercial web search engine as of February 2024.
  arXiv  Detail & Related papers  (2024-11-07T08:54:46Z)
• Efficiency Unleashed: Inference Acceleration for LLM-based Recommender Systems with Speculative Decoding [61.45448947483328]
  We introduce Lossless Acceleration via Speculative Decoding for LLM-based Recommender Systems (LASER)<n>LASER features a Customized Retrieval Pool to enhance retrieval efficiency and Relaxed Verification to improve the acceptance rate of draft tokens.<n>LASER achieves a 3-5x speedup on public datasets and saves about 67% of computational resources during the online A/B test.
  arXiv  Detail & Related papers  (2024-08-11T02:31:13Z)
• FIRST: Faster Improved Listwise Reranking with Single Token Decoding [56.727761901751194]
  First, we introduce FIRST, a novel listwise LLM reranking approach leveraging the output logits of the first generated identifier to directly obtain a ranked ordering of the candidates. Empirical results demonstrate that FIRST accelerates inference by 50% while maintaining a robust ranking performance with gains across the BEIR benchmark. Our results show that LLM rerankers can provide a stronger distillation signal compared to cross-encoders, yielding substantial improvements in retriever recall after relevance feedback.
  arXiv  Detail & Related papers  (2024-06-21T21:27:50Z)
• Aligning Language Models with Demonstrated Feedback [58.834937450242975]
  Demonstration ITerated Task Optimization (DITTO) directly aligns language model outputs to a user's demonstrated behaviors.<n>We evaluate DITTO's ability to learn fine-grained style and task alignment across domains such as news articles, emails, and blog posts.
  arXiv  Detail & Related papers  (2024-06-02T23:13:56Z)
• C-ICL: Contrastive In-context Learning for Information Extraction [54.39470114243744]
  c-ICL is a novel few-shot technique that leverages both correct and incorrect sample constructions to create in-context learning demonstrations. Our experiments on various datasets indicate that c-ICL outperforms previous few-shot in-context learning methods.
  arXiv  Detail & Related papers  (2024-02-17T11:28:08Z)
• LLMRefine: Pinpointing and Refining Large Language Models via Fine-Grained Actionable Feedback [65.84061725174269]
  Recent large language models (LLM) are leveraging human feedback to improve their generation quality. We propose LLMRefine, an inference time optimization method to refine LLM's output. We conduct experiments on three text generation tasks, including machine translation, long-form question answering (QA), and topical summarization. LLMRefine consistently outperforms all baseline approaches, achieving improvements up to 1.7 MetricX points on translation tasks, 8.1 ROUGE-L on ASQA, 2.2 ROUGE-L on topical summarization.
  arXiv  Detail & Related papers  (2023-11-15T19:52:11Z)
• Parrot: Enhancing Multi-Turn Instruction Following for Large Language Models [79.32652077838046]
  We introduce Parrot, a solution aiming to enhance multi-turn instruction following for large language models (LLMs) First, we introduce an efficient but effective method for collecting multi-turn instructions that feature human-like queries, such as anaphora and ellipsis. Second, we propose a context-aware preference optimization strategy to further enhance LLMs for complex queries in multi-turn interaction.
  arXiv  Detail & Related papers  (2023-10-11T08:36:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.