CTR-Driven Ad Text Generation via Online Feedback Preference Optimization

• URL: http://arxiv.org/abs/2507.20227v3
• Date: Sat, 02 Aug 2025 07:32:08 GMT
• Title: CTR-Driven Ad Text Generation via Online Feedback Preference Optimization
• Authors: Yanda Chen, Zihui Ren, Qixiang Gao, Jiale Chen, Si Chen, Xubin Li, Tiezheng Ge, Bo Zheng,
• Abstract summary: Large Language Models (LLMs) offer efficiency advantages over manual ad text creation.<n>LLMs do not guarantee higher CTR performance compared to human-crafted texts.<n>We propose a novel ad text generation method which optimize for CTR through preference optimization from online feedback.
• Score: 28.734264007257085
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Advertising text plays a critical role in determining click-through rates (CTR) in online advertising. Large Language Models (LLMs) offer significant efficiency advantages over manual ad text creation. However, LLM-generated ad texts do not guarantee higher CTR performance compared to human-crafted texts, revealing a gap between generation quality and online performance of ad texts. In this work, we propose a novel ad text generation method which optimizes for CTR through preference optimization from online feedback. Our approach adopts an innovative two-stage framework: (1) diverse ad text sampling via one-shot in-context learning, using retrieval-augmented generation (RAG) to provide exemplars with chain-of-thought (CoT) reasoning; (2) CTR-driven preference optimization from online feedback, which weighs preference pairs according to their CTR gains and confidence levels. Through our method, the resulting model enables end-to-end generation of high-CTR ad texts. Extensive experiments have demonstrated the effectiveness of our method in both offline and online metrics. Notably, we have applied our method on a large-scale online shopping platform and achieved significant CTR improvements, showcasing its strong applicability and effectiveness in advertising systems.

Related papers

• Improving Generative Ad Text on Facebook using Reinforcement Learning [11.28110246872973]
  We present the first deployment of an RL-trained large language model for generative advertising on Facebook.<n>Our model, "AdLlama," powers an AI tool that helps advertisers create new variations of human-written ad text.<n>In a large-scale 10-week A/B test on Facebook, we find that AdLlama improves click-through rates by 6.7% compared to a supervised imitation model trained on curated ads.
  arXiv  Detail & Related papers  (2025-07-29T16:34:02Z)
• CTR-Driven Advertising Image Generation with Multimodal Large Language Models [53.40005544344148]
  We explore the use of Multimodal Large Language Models (MLLMs) for generating advertising images by optimizing for Click-Through Rate (CTR) as the primary objective.<n>To further improve the CTR of generated images, we propose a novel reward model to fine-tune pre-trained MLLMs through Reinforcement Learning (RL)<n>Our method achieves state-of-the-art performance in both online and offline metrics.
  arXiv  Detail & Related papers  (2025-02-05T09:06:02Z)
• SoPo: Text-to-Motion Generation Using Semi-Online Preference Optimization [82.83603957387442]
  We focus on fine-tuning text-to-motion models to consistently favor high-quality, human-preferred motions.<n>We introduce Semi-online Preference Optimization (SoPo), a DPO-based method for training text-to-motion models.<n>Experiments demonstrate that SoPo outperforms other preference alignment methods.
  arXiv  Detail & Related papers  (2024-12-06T14:50:38Z)
• Bridging SFT and DPO for Diffusion Model Alignment with Self-Sampling Preference Optimization [67.8738082040299]
  Self-Sampling Preference Optimization (SSPO) is a new alignment method for post-training reinforcement learning.<n>SSPO eliminates the need for paired data and reward models while retaining the training stability of SFT.<n>SSPO surpasses all previous approaches on the text-to-image benchmarks and demonstrates outstanding performance on the text-to-video benchmarks.
  arXiv  Detail & Related papers  (2024-10-07T17:56:53Z)
• SAIL: Self-Improving Efficient Online Alignment of Large Language Models [56.59644677997827]
  Reinforcement Learning from Human Feedback is a key method for aligning large language models with human preferences. Recent literature has focused on designing online RLHF methods but still lacks a unified conceptual formulation. Our approach significantly improves alignment performance on open-sourced datasets with minimal computational overhead.
  arXiv  Detail & Related papers  (2024-06-21T18:05:35Z)
• CELA: Cost-Efficient Language Model Alignment for CTR Prediction [70.65910069412944]
  Click-Through Rate (CTR) prediction holds a paramount position in recommender systems.<n>Recent efforts have sought to mitigate these challenges by integrating Pre-trained Language Models (PLMs)<n>We propose textbfCost-textbfEfficient textbfLanguage Model textbfAlignment (textbfCELA) for CTR prediction.
  arXiv  Detail & Related papers  (2024-05-17T07:43:25Z)
• Parallel Ranking of Ads and Creatives in Real-Time Advertising Systems [20.78133992969317]
  We propose for the first time a novel architecture for online parallel estimation of ads and creatives ranking. The online architecture enables sophisticated personalized creative modeling while reducing overall latency. The offline joint model for CTR estimation allows mutual awareness and collaborative optimization between ads and creatives.
  arXiv  Detail & Related papers  (2023-12-20T04:05:21Z)
• CREATER: CTR-driven Advertising Text Generation with Controlled Pre-Training and Contrastive Fine-Tuning [14.912117221662054]
  We propose CREATER, a CTR-driven advertising text generation approach, to generate ad texts based on high-quality user reviews. To incorporate CTR objective, our model learns from online A/B test data with contrastive learning, which encourages the model to generate ad texts that obtain higher CTR. Experiments on industrial datasets show that CREATER significantly outperforms current approaches.
  arXiv  Detail & Related papers  (2022-05-18T14:17:04Z)
• TSI: an Ad Text Strength Indicator using Text-to-CTR and Semantic-Ad-Similarity [16.10904771281746]
  We propose an ad text strength indicator (TSI) which: (i) predicts the click-through-rate (CTR) for an input ad text, (ii) fetches similar existing ads to create a neighborhood around the input ad, and compares the predicted CTRs in the neighborhood to declare whether the input ad is strong or weak. As suggestions for ad text improvement, TSI shows anonymized versions of superior ads (higher predicted CTR) in the neighborhood.
  arXiv  Detail & Related papers  (2021-08-18T16:24:40Z)
• Learning to Create Better Ads: Generation and Ranking Approaches for Ad Creative Refinement [26.70647666598025]
  We study approaches to refine the given ad text and image by: (i) generating new ad text, (ii) recommending keyphrases for new ad text, and (iii) recommending image tags (objects in image) Based on A/B tests conducted by multiple advertisers, we form pairwise examples of inferior and superior ad creatives. We also share broadly applicable insights from our experiments using data from the Yahoo Gemini ad platform.
  arXiv  Detail & Related papers  (2020-08-17T16:46:28Z)
• AliExpress Learning-To-Rank: Maximizing Online Model Performance without Going Online [60.887637616379926]
  This paper proposes an evaluator-generator framework for learning-to-rank. It consists of an evaluator that generalizes to evaluate recommendations involving the context, and a generator that maximizes the evaluator score by reinforcement learning. Our method achieves a significant improvement in terms of Conversion Rate (CR) over the industrial-level fine-tuned model in online A/B tests.
  arXiv  Detail & Related papers  (2020-03-25T10:27:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.