CORAL: An LLM-Native Harness for Production Recommender Systems

• URL: http://arxiv.org/abs/2609.02730v1
• Date: Wed, 02 Sep 2026 15:40:36 GMT
• Title: CORAL: An LLM-Native Harness for Production Recommender Systems
• Abstract summary: We present CORAL (Constraint-d Recommender via an Agentic Loop), an agentic loop that learns from the measured effects of its decisions.<n>We formulate this as a partially observed, non-stationary, constrained optimization problem in which the policy improves in context.<n>Across two large-scale social platforms, evaluated with A/B experiments, the same harness improves engagement at no additional serving cost on one and reduces serving cost without degrading engagement on the other.
• Score: 26.7379632070749
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Production recommender systems shape what billions of people see, and sustaining their performance requires continual optimization: as content, user behavior, and upstream models shift, the choices governing retrieval, ranking, and serving must be revisited. Traditionally, human engineers test such changes through online experiments--a slow, reactive process limited by engineering effort, leaving parts of the system unrevised as conditions change. Although large language models have been applied to ranking, user modeling, and offline model development, few systems place an agent in a continual closed loop that acts on a live recommender and learns from the measured effects of its decisions. We present CORAL (Constraint-Optimized Recommender via an Agentic Loop), an LLM-native harness that closes this loop: each cycle, the agent observes operating signals, reasons over a memory of past decisions and outcomes, and invokes tools--including a numerical optimizer that keeps changes within a fixed operating budget--to reconfigure the recommender, with measured outcomes informing the next cycle. We formulate this as a partially observed, non-stationary, constrained optimization problem in which the policy improves in context, without parameter updates, from its prior actions. Across two large-scale social platforms, evaluated with A/B experiments, the same harness improves engagement at no additional serving cost on one and reduces serving cost without degrading engagement on the other, spanning the engagement-efficiency frontier. Performance improves as the loop iterates, suggesting that a single agentic loop can automate continual optimization work traditionally performed by human algorithm engineers under explicit guardrails.

Related papers

• The Optimizer Is the Agent: Reasoning-Driven Search across Prompts, Programs, and ML Workflows [72.9051182568928]
  We present ReASearch, a unified framework for reasoning-driven optimization.<n>With a shared agent loop and domain-specific tools, ReASearch instantiates the exact same scaffold to optimize prompts, programs, and ML.<n>We observe that complex search behaviors emerge naturally from the agent's reasoning process.
  arXiv  Detail & Related papers  (2026-08-07T02:15:39Z)
• OPTScientist: Multi-Agent Discovery of Typed Optimizer Programs for Transformer Pretraining [22.097466841331258]
  We introduce OPTScientist, a theory-guided multi-agent framework for discovery in a typed domain-specific language.<n>Four role agents, Theorist, Designer, Engineer, and Reviewer, collaborate within a single orchestration loop to propose, hypotheses, compile, evaluate, and critique results.<n>Our results suggest a path toward automated science grounded in theory, typed programs, compiler validation, and closed-loop experimentation.
  arXiv  Detail & Related papers  (2026-06-02T15:21:53Z)
• Democratizing Large-Scale Re-Optimization with LLM-Guided Model Patches [14.24802309734814]
  This paper introduces an agentic re-optimization framework in which a large language model (LLM) acts as an OR expert.<n>The proposed framework enables interactive and continuous adaptation of deployed optimization models.
  arXiv  Detail & Related papers  (2026-05-18T17:28:25Z)
• OLIVIA: Online Learning via Inference-time Action Adaptation for Decision Making in LLM ReAct Agents [74.20327254615854]
  Large language model agents interleave reasoning, action selection, and observation to solve sequential decision-making tasks.<n>Existing inference-time adaptation methods for LLM agents mainly rely on prompting or retrieval.<n>We propose OLIVIA, an inference-time action adaptation framework for ReAct-style agents.
  arXiv  Detail & Related papers  (2026-05-11T19:28:20Z)
• A Production-Ready RL Framework for Personalized Utility Tuning with Pareto Sweeping in Pinterest Recommender Systems [7.4144229390766005]
  Large-scale recommenders encode multi-objective trade-offs by combining multiple predicted outcomes into a single utility score.<n>We propose PRL-PUTS, a Production-ready, ranker independent RL framework for personalized utility-weight tuning.
  arXiv  Detail & Related papers  (2026-05-08T01:48:49Z)
• Execution-Verified Reinforcement Learning for Optimization Modeling [49.171122807323634]
  Execution-verified learning framework treats a mathematical programming solver as a deterministic, interactive verifier.<n>Experiments on NL4OPT, MAMO, IndustryOR, and OptiBench across Gurobi, OR-Tools, and COPT show that EVOM matches or outperforms process-supervised SFT.
  arXiv  Detail & Related papers  (2026-04-01T03:39:11Z)
• Constraint-Aware Generative Re-ranking for Multi-Objective Optimization in Advertising Feeds [3.791872390898376]
  We propose a constraint-aware generative reranking framework that transforms constrained optimization into bounded neural decoding.<n>Our framework unifies sequence generation and reward estimation into a single network.<n> Experiments on large-scale industrial feeds and online A/B tests show that our method improves revenue and user engagement while meeting strict latency requirements.
  arXiv  Detail & Related papers  (2026-03-04T16:09:36Z)
• RecNet: Self-Evolving Preference Propagation for Agentic Recommender Systems [109.9061591263748]
  RecNet is a self-evolving preference propagation framework for recommender systems.<n>It proactively propagates real-time preference updates across related users and items.<n>In the backward phase, the feedback-driven propagation optimization mechanism simulates a multi-agent reinforcement learning framework.
  arXiv  Detail & Related papers  (2026-01-29T12:14:31Z)
• LLM-guided Chemical Process Optimization with a Multi-Agent Approach [8.714038047141202]
  We present a multi-agent LLM framework that autonomously infers operating constraints from minimal process descriptions.<n>Our AutoGen-based framework employs OpenAI's o3 model with specialized agents for constraint generation, parameter validation, simulation, and optimization guidance.
  arXiv  Detail & Related papers  (2025-06-26T01:03:44Z)
• Preference Optimization for Combinatorial Optimization Problems [54.87466279363487]
  Reinforcement Learning (RL) has emerged as a powerful tool for neural optimization, enabling models learns that solve complex problems without requiring expert knowledge.<n>Despite significant progress, existing RL approaches face challenges such as diminishing reward signals and inefficient exploration in vast action spaces.<n>We propose Preference Optimization, a novel method that transforms quantitative reward signals into qualitative preference signals via statistical comparison modeling.
  arXiv  Detail & Related papers  (2025-05-13T16:47:00Z)
• On the Role of Feedback in Test-Time Scaling of Agentic AI Workflows [71.92083784393418]
  Agentic AI (systems that autonomously plan and act) are becoming widespread, yet their task success rate on complex tasks remains low.<n>Inference-time alignment relies on three components: sampling, evaluation, and feedback.<n>We introduce Iterative Agent Decoding (IAD), a procedure that repeatedly inserts feedback extracted from different forms of critiques.
  arXiv  Detail & Related papers  (2025-04-02T17:40:47Z)
• Dynamic Rewarding with Prompt Optimization Enables Tuning-free Self-Alignment of Language Models [54.381650481255235]
  We introduce a new tuning-free approach for self-alignment, Dynamic Rewarding with Prompt Optimization (O) Our approach leverages a search-based optimization framework that allows LLMs to iteratively self-improve and craft the optimal alignment instructions. Empirical evaluations on eight recent LLMs, both open and closed-sourced, demonstrate that DRPO significantly enhances alignment performance.
  arXiv  Detail & Related papers  (2024-11-13T16:15:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.