User Feedback Provides a Unique Signal that LLMs Can not Detect

• URL: http://arxiv.org/abs/2609.02859v1
• Date: Wed, 02 Sep 2026 17:42:44 GMT
• Title: User Feedback Provides a Unique Signal that LLMs Can not Detect
• Abstract summary: We demonstrate that user feedback is a highly actionable signal for improvement.<n>We show that feedback-informed revisions resolve targeted issues at significantly higher rates than baseline revisions.
• Score: 25.849928795741672
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Harnessing naturally occurring feedback from user interactions offers a promising learning signal for Large Language Models (LLMs). However, recent studies suggest this feedback is inherently noisy and difficult to leverage effectively. We challenge this conception by demonstrating that user feedback is a highly actionable signal for improvement, and that its perceived ineffectiveness stems from a systematic bias in current evaluation paradigms. To isolate the usefulness of feedback, we construct synthetic data with a definitive ground truth, alongside naturalistic data to validate that our findings hold in real-world scenarios. By comparing model revisions generated with and without access to feedback across both settings, we show that feedback-informed revisions resolve targeted issues at significantly higher rates than baseline revisions. Finally, we expose the root of the evaluation bias: when a model successfully fixes an issue exclusively due to feedback, LLM judges frequently fail to identify the genuinely corrected response, systematically preferring inferior baseline outputs instead.

Related papers

• CausalRM: Causal-Theoretic Reward Modeling for RLHF from Observational User Feedbacks [65.44788139573144]
  We introduce observational reward modeling -- learning reward models with observational user feedback -- as a scalable and cost-effective alternative.<n>CaulRM aims to learn unbiased reward models from observational feedback.<n>Experiments validate that CausalRM effectively learns accurate reward signals from noisy and biased observational feedback.
  arXiv  Detail & Related papers  (2026-03-19T10:37:34Z)
• CoNRec: Context-Discerning Negative Recommendation with LLMs [5.832474387562381]
  Research into users' negative preferences has gained increasing importance in modern recommendation systems.<n>Most existing approaches primarily use negative feedback as an auxiliary signal to enhance positive recommendations.<n>We propose the first large language model framework for negative feedback modeling with special designed context-discerning modules.
  arXiv  Detail & Related papers  (2026-01-22T07:46:18Z)
• In-Place Feedback: A New Paradigm for Guiding LLMs in Multi-Turn Reasoning [10.138497038893096]
  We introduce in-place feedback, a novel interaction paradigm in which users directly edit an LLM's previous response.<n> Empirical evaluations on reasoning-intensive benchmarks reveal that in-place feedback achieves better performance than conventional multi-turn feedback.
  arXiv  Detail & Related papers  (2025-10-01T11:16:04Z)
• Critique to Verify: Accurate and Honest Test-Time Scaling with RL-Trained Verifiers [63.99316853136304]
  Mirror-Critique is a framework that trains a verifier with informative critiques.<n>We deploy a small instruction-tuned model to synthesize high-quality critique data.<n>The resulting Mirror-Verifier is deployed to evaluate candidate solutions by generating multiple critiques per solution.
  arXiv  Detail & Related papers  (2025-09-27T06:50:24Z)
• WildFeedback: Aligning LLMs With In-situ User Interactions And Feedback [36.06000681394939]
  We introduce WildFeedback, a novel framework that leverages in-situ user feedback during conversations with large language models (LLMs) to create preference datasets automatically.<n>Our experiments demonstrate that LLMs fine-tuned on WildFeedback dataset exhibit significantly improved alignment with user preferences.
  arXiv  Detail & Related papers  (2024-08-28T05:53:46Z)
• Is user feedback always informative? Retrieval Latent Defending for Semi-Supervised Domain Adaptation without Source Data [34.55109747972333]
  This paper aims to adapt the source model to the target environment using user feedback readily available in real-world applications. We analyze this phenomenon via a novel concept called Negatively Biased Feedback (NBF) We propose a scalable adapting approach, Retrieval Latent Defending.
  arXiv  Detail & Related papers  (2024-07-22T05:15:41Z)
• Beyond Thumbs Up/Down: Untangling Challenges of Fine-Grained Feedback for Text-to-Image Generation [67.88747330066049]
  Fine-grained feedback captures nuanced distinctions in image quality and prompt-alignment. We show that demonstrating its superiority to coarse-grained feedback is not automatic. We identify key challenges in eliciting and utilizing fine-grained feedback.
  arXiv  Detail & Related papers  (2024-06-24T17:19:34Z)
• Rethinking the Evaluation of Dialogue Systems: Effects of User Feedback on Crowdworkers and LLMs [57.16442740983528]
  In ad-hoc retrieval, evaluation relies heavily on user actions, including implicit feedback. The role of user feedback in annotators' assessment of turns in a conversational perception has been little studied. We focus on how the evaluation of task-oriented dialogue systems ( TDSs) is affected by considering user feedback, explicit or implicit, as provided through the follow-up utterance of a turn being evaluated.
  arXiv  Detail & Related papers  (2024-04-19T16:45:50Z)
• RLVF: Learning from Verbal Feedback without Overgeneralization [94.19501420241188]
  We study the problem of incorporating verbal feedback without such overgeneralization. We develop a new method Contextualized Critiques with Constrained Preference Optimization (C3PO) Our approach effectively applies verbal feedback to relevant scenarios while preserving existing behaviors for other contexts.
  arXiv  Detail & Related papers  (2024-02-16T18:50:24Z)
• DPR: An Algorithm Mitigate Bias Accumulation in Recommendation feedback loops [41.21024436158042]
  We study the negative impact of feedback loops and unknown exposure mechanisms on recommendation quality and user experience. We propose Dynamic Personalized Ranking (textbfDPR), an unbiased algorithm that uses dynamic re-weighting to mitigate the cross-effects. We show theoretically that our approach mitigates the negative effects of feedback loops and unknown exposure mechanisms.
  arXiv  Detail & Related papers  (2023-11-10T04:36:00Z)
• Simulating Bandit Learning from User Feedback for Extractive Question Answering [51.97943858898579]
  We study learning from user feedback for extractive question answering by simulating feedback using supervised data. We show that systems initially trained on a small number of examples can dramatically improve given feedback from users on model-predicted answers.
  arXiv  Detail & Related papers  (2022-03-18T17:47:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.