CoRe: A Continuously Reward-Finetuned LLM Query Rewriter for Multi-Stage Context-Aware Relevance in Web-Scale Video Search

• URL: http://arxiv.org/abs/2606.14127v1
• Date: Fri, 12 Jun 2026 05:19:40 GMT
• Title: CoRe: A Continuously Reward-Finetuned LLM Query Rewriter for Multi-Stage Context-Aware Relevance in Web-Scale Video Search
• Abstract summary: We present CoRe, a system that redeployed weekly for over five months in a major short-video search engine.<n>Our reward uses the deployed multimodal relevance model as its source and a multiplicative ratio form mirroring the production fusion algebra.<n>A semi-online Mixed Preference Optimization loop makes this reward affordable at multi-million-instance weekly scale.
• Score: 16.32525191514382
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: LLM-based query rewriters in production face a tension: the training reward must reflect how the rewrite is consumed by the production ranker, yet the training procedure must be cheap enough to support continuous redeployment as data drifts. We present CoRe (Context Relevance), such a system, redeployed weekly for over five months in a major short-video search engine. Our reward uses the deployed multimodal relevance model as its source and a multiplicative ratio form mirroring the production fusion algebra, closing the simulation-production gap that offline reward proxies leave open. A semi-online Mixed Preference Optimization loop makes this reward affordable at multi-million-instance weekly scale: a DPO-style pairwise objective restricts the gradient pass to a small top-k/bottom-k subset of sampled trajectories, and a phase structure reduces trainer/inference-server parameter syncs from per-step to per-phase. An automated promotion gate over reward-like and stability metrics detected and recovered from a real reward-hacking incident in production. Rewriter output is consumed as parallel relevance signals at recall, rawrank, and finerank without displacing the original signals, bounding rewriter-failure blast radius. Online A/B from two sequential production launches, first deploying the rewriter at finerank, then extending consumption to recall and rawrank, delivers statistically significant reductions in change-query rate on rewrite-impacted queries, with all headline relevance and engagement metrics moving in the expected direction.

Related papers

• From Language to Behavior: Scaling Sequence Transformers for Industrial Recommendation Ranking with Rec-Native Designs [20.42858135695384]
  ReST is a recommendation-native Transformer scaling framework.<n>For signal quality, it introduces a sequence encoder with dual-gated attention.<n>For asymmetry, it factorizes ranking into a heavy reusable encoder and decoder.
  arXiv  Detail & Related papers  (2026-09-01T13:40:36Z)
• ResiSpec: Enhancing Multi-Candidate Speculative Sampling via Residual Distribution Shaping [79.1905186547363]
  ResiSpec is a framework that strategically reforms the proposal distribution during verification.<n>It prevents candidate obsolescence and achieves up to 1.92$times$ speedup over state-of-the-art multi-candidate methods.
  arXiv  Detail & Related papers  (2026-08-25T11:25:55Z)
• Deep-learning Causal Retrieval Optimization for Efficient e-commerce Distribution in Pinterest [0.7827524915894094]
  We deploy a production system at Pinterest that learns personalized and contextualized triggering policies.<n>A deep multi-task model jointly predicts outcomes and uplift of multiple events.<n>A linear-time offline replay is designed to select thresholds and forecast policy impact.
  arXiv  Detail & Related papers  (2026-07-14T23:02:27Z)
• Latent Recurrent Transformer: Architecture Exploration, Training Strategies, and Scaling Behavior [107.2098567818173]
  Latent Recurrent Transformer (LRT) is a lightweight augmentation of autoregressive transformers.<n>LRT reuses a high-level source-layer hidden state from the previous token as recurrent memory for the next token.
  arXiv  Detail & Related papers  (2026-05-26T10:10:26Z)
• RAVEN: Real-time Autoregressive Video Extrapolation with Consistency-model GRPO [53.38929612273108]
  Causal autoregressive video diffusion models support real-time streaming generation by extrapolating future chunks from previously generated content.<n>We introduce the Real-time Autoregressive Video Extrapolation Network (RAVEN), a training-time test framework that repacks each self rollout into an interleaved sequence of clean historical endpoints.
  arXiv  Detail & Related papers  (2026-05-14T17:59:30Z)
• REALM: Retrospective Encoder Alignment for LFP Modeling [2.8036525859094823]
  Spike activity has been the dominant neural signal for behavior decoding due to its high spatial and temporal resolution.<n>As brain-computer interfaces (BCIs) move toward high channel counts and wireless operation, the high sampling frequency of spike signals becomes a bottleneck.<n>We propose REALM: a retrospective distillation framework that enables causal LFP decoding.
  arXiv  Detail & Related papers  (2026-05-14T14:16:22Z)
• CreFlow: Corrective Reflow for Sparse-Reward Embodied Video Diffusion RL [56.969946199335716]
  We introduce a compositional constraint-based reward model for post-training embodied video generation models.<n>We propose CreFlow, a novel online RL framework with two key designs.
  arXiv  Detail & Related papers  (2026-05-14T02:18:58Z)
• ReCoVer: Resilient LLM Pre-Training System via Fault-Tolerant Collective and Versatile Workload [18.933354857797056]
  Pre-training large language models on massive GPU clusters has made hardware faults routine rather than rare.<n>We propose ReCoVer, a resilient pre-training system that upholds a single invariant.<n>For comparison with checkpoint-and-restart baselines, ReCoVer demonstrates $2.23times$ higher effective throughput after successive failures.
  arXiv  Detail & Related papers  (2026-05-11T20:28:31Z)
• ResRank: Unifying Retrieval and Listwise Reranking via End-to-End Joint Training with Residual Passage Compression [5.223477157195793]
  "lost in the middle" phenomenon degrades ranking quality as input length grows.<n>Inference latency scales super-linearly with sequence length, rendering listwise reranking impractical for industrial deployment.<n>We present ResRank, a unified retrieval-reranking framework that fundamentally addresses both challenges.
  arXiv  Detail & Related papers  (2026-04-24T03:11:51Z)
• GenRec: A Preference-Oriented Generative Framework for Large-Scale Recommendation [14.663434490160016]
  We present GenRec, a preference-oriented generative framework deployed on the JD App.<n>In month-long online A/B tests, GenRec achieves 9.5% improvement in click count and 8.7% in transaction count over the existing pipeline.
  arXiv  Detail & Related papers  (2026-04-16T11:07:05Z)
• Astrolabe: Steering Forward-Process Reinforcement Learning for Distilled Autoregressive Video Models [58.3184497327891]
  Distilled autoregressive (AR) video models enable efficient streaming generation but frequently misalign with human visual preferences.<n>We present Astrolabe, an efficient online reinforcement learning framework tailored for distilled AR models.
  arXiv  Detail & Related papers  (2026-03-17T18:32:18Z)
• SoliReward: Mitigating Susceptibility to Reward Hacking and Annotation Noise in Video Generation Reward Models [53.19726629537694]
  Post-training alignment of video generation models with human preferences is a critical goal.<n>Current data collection paradigms, reliant on in-prompt pairwise annotations, suffer from labeling noise.<n>We propose SoliReward, a systematic framework for video RM training.
  arXiv  Detail & Related papers  (2025-12-17T14:28:23Z)
• Regression-aware Inference with LLMs [52.764328080398805]
  We show that an inference strategy can be sub-optimal for common regression and scoring evaluation metrics. We propose alternate inference strategies that estimate the Bayes-optimal solution for regression and scoring metrics in closed-form from sampled responses.
  arXiv  Detail & Related papers  (2024-03-07T03:24:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.