Talking to a Know-It-All GPT or a Second-Guesser Claude? How Repair reveals unreliable Multi-Turn Behavior in LLMs

• URL: http://arxiv.org/abs/2604.19245v2
• Date: Wed, 22 Apr 2026 07:02:47 GMT
• Title: Talking to a Know-It-All GPT or a Second-Guesser Claude? How Repair reveals unreliable Multi-Turn Behavior in LLMs
• Abstract summary: Repair is an important resource for resolving trouble in human-human conversation.<n>This study investigates how LLMs engage in the interactive process of repair in multi-turn dialogues around solvable and unsolvable math questions.
• Score: 10.068355434387845
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Repair, an important resource for resolving trouble in human-human conversation, remains underexplored in human-LLM interaction. In this study, we investigate how LLMs engage in the interactive process of repair in multi-turn dialogues around solvable and unsolvable math questions. We examine whether models initiate repair themselves and how they respond to user-initiated repair. Our results show strong differences across models: reactions range from being almost completely resistant to (appropriate) repair attempts to being highly susceptible and easily manipulated. We further demonstrate that once conversations extend beyond a single turn, model behavior becomes more distinctive and less predictable across systems. Overall, our findings indicate that each tested LLM exhibits its own characteristic form of unreliability in the context of repair.

Related papers

• Found in Conversation: LLMs Teach Themselves to Close the Multi-Turn Gap [51.23637310487053]
  We introduce Found in Conversation, a training framework where a model teaches itself to find and recover its single-turn competence.<n>View-Asymmetric Self-Distillation distills across two views of the same task information--single-turn view for the teacher, multi-turn view for the student--transferring strong single-turn behavior into weak multi-turn behavior.<n>Across model families (Llama, Qwen, Phi, and OLMo), FiC recovers at least 92% of single-turn performance and reaches 100% on two Llama backbones.
  arXiv  Detail & Related papers  (2026-05-23T06:58:52Z)
• "Mm, Wat?" Detecting Other-initiated Repair Requests in Dialogue [1.0616273526777913]
  This work proposes a multimodal model to automatically detect repair initiation in Dutch dialogues.<n>The results show that prosodic cues complement linguistic features and significantly improve the results of pretrained text and audio embeddings.
  arXiv  Detail & Related papers  (2025-10-28T16:58:26Z)
• If an LLM Were a Character, Would It Know Its Own Story? Evaluating Lifelong Learning in LLMs [55.8331366739144]
  We introduce LIFESTATE-BENCH, a benchmark designed to assess lifelong learning in large language models (LLMs)<n>Our fact checking evaluation probes models' self-awareness, episodic memory retrieval, and relationship tracking, across both parametric and non-parametric approaches.
  arXiv  Detail & Related papers  (2025-03-30T16:50:57Z)
• An Empirical Study of the Role of Incompleteness and Ambiguity in Interactions with Large Language Models [0.9856777842758593]
  We present a neural symbolic framework that models the interactions between human and Large Language Models (LLMs)<n>We define incompleteness and ambiguity in the questions as properties deducible from the messages exchanged in the interaction.<n>Our results show multi-turn interactions are usually required for datasets which have a high proportion of incompleteness or ambiguous questions.
  arXiv  Detail & Related papers  (2025-03-23T04:34:30Z)
• Studying and Understanding the Effectiveness and Failures of Conversational LLM-Based Repair [3.93048798243871]
  Automated program repair (APR) is designed to automate the process of bug-fixing.<n>Advanced APR techniques powered by conversational language models (LLMs) have exhibited impressive repair abilities.<n>Despite the superiority, conversational APR techniques still fail to repair a large number of bugs.
  arXiv  Detail & Related papers  (2025-03-19T09:39:32Z)
• MMRC: A Large-Scale Benchmark for Understanding Multimodal Large Language Model in Real-World Conversation [52.35744453954844]
  This paper introduces MMRC, a benchmark for evaluating six core open-ended abilities of MLLMs.<n> Evaluations on 20 MLLMs in MMRC indicate an accuracy drop during open-ended interactions.<n>We propose a simple yet effective NOTE-TAKING strategy, which can record key information from the conversation and remind the model during its responses.
  arXiv  Detail & Related papers  (2025-02-17T15:24:49Z)
• Repairs in a Block World: A New Benchmark for Handling User Corrections with Multi-Modal Language Models [48.42142115255159]
  We release BlockWorld-Repairs: a dataset of multi-modal TPR sequences in an instruction-following manipulation task. We evaluate several state-of-the-art Vision and Language Models (VLM) across multiple settings, focusing on their capability to process and accurately respond to TPRs. Our results suggest that these models are not yet ready to be deployed in multi-modal collaborative settings.
  arXiv  Detail & Related papers  (2024-09-21T21:06:25Z)
• Retrieval-enhanced Knowledge Editing in Language Models for Multi-Hop Question Answering [47.199078631274745]
  Large Language Models (LLMs) have shown proficiency in question-answering tasks but often struggle to integrate real-time knowledge. We propose the Retrieval-Augmented model Editing (RAE) framework for multi-hop question answering.
  arXiv  Detail & Related papers  (2024-03-28T17:47:19Z)
• ReST meets ReAct: Self-Improvement for Multi-Step Reasoning LLM Agent [50.508669199496474]
  We develop a ReAct-style LLM agent with the ability to reason and act upon external knowledge. We refine the agent through a ReST-like method that iteratively trains on previous trajectories. Starting from a prompted large model and after just two iterations of the algorithm, we can produce a fine-tuned small model.
  arXiv  Detail & Related papers  (2023-12-15T18:20:15Z)
• FreshLLMs: Refreshing Large Language Models with Search Engine Augmentation [92.43001160060376]
  We study the factuality of large language models (LLMs) in the context of answering questions that test current world knowledge. We introduce FreshQA, a novel dynamic QA benchmark encompassing a diverse range of question and answer types. We benchmark a diverse array of both closed and open-source LLMs under a two-mode evaluation procedure that allows us to measure both correctness and hallucination. Motivated by these results, we present FreshPrompt, a simple few-shot prompting method that substantially boosts the performance of an LLM on FreshQA.
  arXiv  Detail & Related papers  (2023-10-05T00:04:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.