Learning from Online User Feedback for Shopping Agents

• URL: http://arxiv.org/abs/2608.11604v1
• Date: Wed, 12 Aug 2026 03:24:44 GMT
• Title: Learning from Online User Feedback for Shopping Agents
• Abstract summary: Large language model-based shopping agents are increasingly deployed in real-world e-commerce platforms.<n>Existing approaches rely on offline training signals, such as user-item interactions or synthetic preference data.<n>We propose LOFA, a framework that enables shopping agents to learn directly from real online interaction logs without human annotation.
• Score: 54.03311028861336
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language model-based shopping agents are increasingly deployed in real-world e-commerce platforms, generating massive amounts of user interaction logs that provide valuable supervision for improving these agents. However, existing approaches primarily rely on offline training signals, such as user-item interactions or synthetic preference data, while largely overlooking the rich supervision contained in users' natural conversational feedback. Moreover, the available online feedback is heterogeneous, sparse, and noisy, making it difficult to transform into reliable learning signals automatically. To address these challenges, we propose LOFA, a framework that enables shopping agents to learn directly from real online interaction logs without human annotation. LOFA combines reinforcement learning over verifiable purchase outcomes with feedback-aware on-policy distillation, which identifies users'in-dialogue directives and converts them into dense token-level supervision. These complementary objectives capture both collaborative behavioral patterns and user-specific preferences. Extensive experiments on real-world e-commerce logs demonstrate that LOFA consistently improves recommendation quality, response helpfulness, and user-satisfaction alignment over strong baselines, highlighting the effectiveness of learning shopping agents from real online user feedback.

Related papers

• User Feedback Provides a Unique Signal that LLMs Can not Detect [25.849928795741672]
  We demonstrate that user feedback is a highly actionable signal for improvement.<n>We show that feedback-informed revisions resolve targeted issues at significantly higher rates than baseline revisions.
  arXiv  Detail & Related papers  (2026-09-02T17:42:44Z)
• World Model-Guided Reinforcement Learning via Counterfactual User Engagement Simulation [43.27233360158595]
  Reinforcement learning for user-centric agents is limited by the cost, latency, and risk of collecting online feedback.<n>We propose World Model-Guided Reinforcement Learning via counterfactual user engagement simulation (WMG-RL)<n>We show that UEWM provides reliable and transferable reward signals across domains.
  arXiv  Detail & Related papers  (2026-09-01T10:59:41Z)
• REFINE: Real-world Exploration of Interactive Feedback and Student Behaviour [12.059739840538638]
  We introduce REFINE, a locally deployable, multi-agent feedback system that treats feedback as an interactive process.<n> REFINE combines a pedagogically-grounded feedback generation agent with an LLM-as-a-judge-guided regeneration loop using a human-aligned judge.<n>Our findings demonstrate the feasibility and effectiveness of multi-agent, tool-augmented feedback systems for scalable, interactive feedback.
  arXiv  Detail & Related papers  (2026-03-31T01:48:08Z)
• Pushing Forward Pareto Frontiers of Proactive Agents with Behavioral Agentic Optimization [61.641777037967366]
  Proactive large language model (LLM) agents aim to actively plan, query, and interact over multiple turns.<n>Agentic reinforcement learning (RL) has emerged as a promising solution for training such agents in multi-turn settings.<n>We propose BAO, an agentic RL framework that combines behavior enhancement to enrich proactive reasoning and information-gathering capabilities.
  arXiv  Detail & Related papers  (2026-02-11T20:40:43Z)
• RecoWorld: Building Simulated Environments for Agentic Recommender Systems [55.979427290369216]
  We present RecoWorld, a blueprint for building simulated environments tailored to agentic recommender systems.<n>A user simulator reviews recommended items, updates its mindset, and when sensing potential user disengagement, generates reflective instructions.<n>The agentic recommender adapts its recommendations by incorporating these user instructions and reasoning traces, creating a dynamic feedback loop.
  arXiv  Detail & Related papers  (2025-09-12T16:44:34Z)
• Reinforcement Learning from User Feedback [28.335218244885706]
  We introduce Reinforcement Learning from User Feedback (RLUF), a framework for aligning large language models with user preferences.<n>We train a reward model, P[Love], to predict the likelihood that an LLM response will receive a Love Reaction.<n>We show that P[Love] is predictive of increased positive feedback and serves as a reliable offline evaluator of future user behavior.
  arXiv  Detail & Related papers  (2025-05-20T22:14:44Z)
• Enhancing User-Oriented Proactivity in Open-Domain Dialogues with Critic Guidance [35.15965694815852]
  Open-domain dialogue systems aim to generate natural and engaging conversations.<n>Existing large language models (LLMs) fall short in proactively understanding the user's chatting preferences.<n>We propose a User-oriented Proactive (UPC) to enhance the user-oriented proactivity.
  arXiv  Detail & Related papers  (2025-05-18T09:59:22Z)
• Reinforced Interactive Continual Learning via Real-time Noisy Human Feedback [59.768119380109084]
  This paper introduces an interactive continual learning paradigm where AI models dynamically learn new skills from real-time human feedback.<n>We propose RiCL, a Reinforced interactive Continual Learning framework leveraging Large Language Models (LLMs)<n>Our RiCL approach substantially outperforms existing combinations of state-of-the-art online continual learning and noisy-label learning methods.
  arXiv  Detail & Related papers  (2025-05-15T03:22:03Z)
• Search-Based Interaction For Conversation Recommendation via Generative Reward Model Based Simulated User [117.82681846559909]
  Conversational recommendation systems (CRSs) use multi-turn interaction to capture user preferences and provide personalized recommendations.<n>We propose a generative reward model based simulated user, named GRSU, for automatic interaction with CRSs.
  arXiv  Detail & Related papers  (2025-04-29T06:37:30Z)
• Trust-Oriented Adaptive Guardrails for Large Language Models [9.719986610417441]
  Guardrails are designed to ensure that large language models (LLMs) align with human values by moderating harmful or toxic responses.<n>This paper addresses a critical issue: existing guardrails lack a well-founded methodology to accommodate the diverse needs of different user groups.<n>We introduce an adaptive guardrail mechanism, to dynamically moderate access to sensitive content based on user trust metrics.
  arXiv  Detail & Related papers  (2024-08-16T18:07:48Z)
• Continual Learning for Instruction Following from Realtime Feedback [23.078048024461264]
  We propose and deploy an approach to continually train an instruction-following agent from feedback provided by users during collaborative interactions. During interaction, human users instruct an agent using natural language, and provide realtime binary feedback as they observe the agent following their instructions. We design a contextual bandit learning approach, converting user feedback to immediate reward. We evaluate through thousands of human-agent interactions, demonstrating 15.4% absolute improvement in instruction execution accuracy over time.
  arXiv  Detail & Related papers  (2022-12-19T18:39:43Z)
• PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-training [94.87393610927812]
  We present an off-policy, interactive reinforcement learning algorithm that capitalizes on the strengths of both feedback and off-policy learning. We demonstrate that our approach is capable of learning tasks of higher complexity than previously considered by human-in-the-loop methods.
  arXiv  Detail & Related papers  (2021-06-09T14:10:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.