What Drives Interactive Improvement from Feedback?

• URL: http://arxiv.org/abs/2606.30774v1
• Date: Mon, 29 Jun 2026 18:06:36 GMT
• Title: What Drives Interactive Improvement from Feedback?
• Abstract summary: We study when natural-language feedback produces improvement beyond the gains obtainable from repeated attempts alone.<n>In multi-turn language agent setting, higher final accuracy can reflect useful feedback, but it can also arise from resampling, format correction, or additional test-time computation.<n>We introduce a controlled student-teacher protocol across Omni-MATH, Codeforces, BBEH Linguini, and ARC-AGI1.
• Score: 0.7330217643497284
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We study when natural-language feedback produces improvement beyond the gains obtainable from repeated attempts alone. In multi-turn language agent setting, higher final accuracy can reflect useful feedback, but it can also arise from resampling, format correction, or additional test-time computation. To separate these effects, we introduce a controlled student-teacher protocol across Omni-MATH, Codeforces, BBEH Linguini, and ARC-AGI1, evaluating thirteen open-weight models in both student and teacher roles. We compare external feedback, self-feedback, and unguided self-refinement, while varying interaction history, task difficulty, and teacher access to privileged task information. Across settings, we find that multi-turn improvement is often not evidence of feedback use: self-generated feedback adds little beyond unguided self-refinement, whereas the strongest external teachers produce substantially larger feedback-specific gains, suggesting that useful feedback must provide guidance beyond generic retry. Dense student-teacher interaction matrices further show that interactive gains are driven more by the student's ability to use feedback than by the teacher's identity, although teacher choice remains important for a fixed student. These results suggest that feedback-based agents should be evaluated against repeated-attempt baselines, and that ability to act on feedback, not merely feedback availability, is a central bottleneck for interactive improvement. We release our controlled student-teacher evaluation framework at https://j-lojek.github.io/feedback-generation-is-a-bottleneck/.

Related papers

• REFINE: Real-world Exploration of Interactive Feedback and Student Behaviour [12.059739840538638]
  We introduce REFINE, a locally deployable, multi-agent feedback system that treats feedback as an interactive process.<n> REFINE combines a pedagogically-grounded feedback generation agent with an LLM-as-a-judge-guided regeneration loop using a human-aligned judge.<n>Our findings demonstrate the feasibility and effectiveness of multi-agent, tool-augmented feedback systems for scalable, interactive feedback.
  arXiv  Detail & Related papers  (2026-03-31T01:48:08Z)
• Humanizing Automated Programming Feedback: Fine-Tuning Generative Models with Student-Written Feedback [21.114005575615586]
  We explore learnersourcing as a means to fine-tune language models for generating feedback that is more similar to that written by humans.<n>We collected approximately 1,900 instances of student-written feedback on multiple programming problems and buggy programs.<n>Our findings indicate that fine-tuning models on learnersourced data not only produces feedback that better matches the style of feedback written by students, but also improves accuracy compared to feedback generated through prompt engineering alone.
  arXiv  Detail & Related papers  (2025-09-12T19:23:05Z)
• CANDERE-COACH: Reinforcement Learning from Noisy Feedback [12.232688822099325]
  The CANDERE-COACH algorithm is capable of learning from noisy feedback by a nonoptimal teacher. We propose a noise-filtering mechanism to de-noise online feedback data, thereby enabling the RL agent to successfully learn with up to 40% of the teacher feedback being incorrect.
  arXiv  Detail & Related papers  (2024-09-23T20:14:12Z)
• "My Grade is Wrong!": A Contestable AI Framework for Interactive Feedback in Evaluating Student Essays [6.810086342993699]
  This paper introduces CAELF, a Contestable AI Empowered LLM Framework for automating interactive feedback. CAELF allows students to query, challenge, and clarify their feedback by integrating a multi-agent system with computational argumentation. A case study on 500 critical thinking essays with user studies demonstrates that CAELF significantly improves interactive feedback.
  arXiv  Detail & Related papers  (2024-09-11T17:59:01Z)
• Beyond Thumbs Up/Down: Untangling Challenges of Fine-Grained Feedback for Text-to-Image Generation [67.88747330066049]
  Fine-grained feedback captures nuanced distinctions in image quality and prompt-alignment. We show that demonstrating its superiority to coarse-grained feedback is not automatic. We identify key challenges in eliciting and utilizing fine-grained feedback.
  arXiv  Detail & Related papers  (2024-06-24T17:19:34Z)
• Improving the Validity of Automatically Generated Feedback via Reinforcement Learning [46.667783153759636]
  We propose a framework for feedback generation that optimize both correctness and alignment using reinforcement learning (RL)<n>Specifically, we use GPT-4's annotations to create preferences over feedback pairs in an augmented dataset for training via direct preference optimization (DPO)
  arXiv  Detail & Related papers  (2024-03-02T20:25:50Z)
• Is Feedback All You Need? Leveraging Natural Language Feedback in Goal-Conditioned Reinforcement Learning [54.31495290436766]
  We extend BabyAI to automatically generate language feedback from environment dynamics and goal condition success. We modify the Decision Transformer architecture to take advantage of this additional signal. We find that training with language feedback either in place of or in addition to the return-to-go or goal descriptions improves agents' generalisation performance.
  arXiv  Detail & Related papers  (2023-12-07T22:33:34Z)
• Constructive Large Language Models Alignment with Diverse Feedback [76.9578950893839]
  We introduce Constructive and Diverse Feedback (CDF) as a novel method to enhance large language models alignment. We exploit critique feedback for easy problems, refinement feedback for medium problems, and preference feedback for hard problems. By training our model with this diversified feedback, we achieve enhanced alignment performance while using less training data.
  arXiv  Detail & Related papers  (2023-10-10T09:20:14Z)
• UltraFeedback: Boosting Language Models with Scaled AI Feedback [99.4633351133207]
  We present textscUltraFeedback, a large-scale, high-quality, and diversified AI feedback dataset. Our work validates the effectiveness of scaled AI feedback data in constructing strong open-source chat language models.
  arXiv  Detail & Related papers  (2023-10-02T17:40:01Z)
• Learning Online from Corrective Feedback: A Meta-Algorithm for Robotics [24.863665993509997]
  A key challenge in Imitation Learning (IL) is that optimal state actions demonstrations are difficult for the teacher to provide. As an alternative to state action demonstrations, the teacher can provide corrective feedback such as their preferences or rewards. We show that our approach can learn quickly from a variety of noisy feedback.
  arXiv  Detail & Related papers  (2021-04-02T12:42:12Z)
• Facial Feedback for Reinforcement Learning: A Case Study and Offline Analysis Using the TAMER Framework [51.237191651923666]
  We investigate the potential of agent learning from trainers' facial expressions via interpreting them as evaluative feedback. With designed CNN-RNN model, our analysis shows that telling trainers to use facial expressions and competition can improve the accuracies for estimating positive and negative feedback. Our results with a simulation experiment show that learning solely from predicted feedback based on facial expressions is possible.
  arXiv  Detail & Related papers  (2020-01-23T17:50:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.