DialogueReason: Rule-Based RL Sparks Dialogue Reasoning in LLMs

• URL: http://arxiv.org/abs/2505.07049v1
• Date: Sun, 11 May 2025 16:39:58 GMT
• Title: DialogueReason: Rule-Based RL Sparks Dialogue Reasoning in LLMs
• Authors: Yubo Shu, Zhewei Huang, Xin Wu, Chen Hu, Shuchang Zhou, Daxin Jiang,
• Abstract summary: We propose DialogueReason, a reasoning paradigm that uncovers the lost roles in monologue-style reasoning models.<n>Our work consists of an analysis of monologue reasoning patterns and the development of a dialogue-based reasoning approach.
• Score: 54.4857963044859
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose DialogueReason, a reasoning paradigm that uncovers the lost roles in monologue-style reasoning models, aiming to boost diversity and coherency of the reasoning process. Recent advances in RL-based large reasoning models have led to impressive long CoT capabilities and high performance on math and science benchmarks. However, these reasoning models rely mainly on monologue-style reasoning, which often limits reasoning diversity and coherency, frequently recycling fixed strategies or exhibiting unnecessary shifts in attention. Our work consists of an analysis of monologue reasoning patterns and the development of a dialogue-based reasoning approach. We first introduce the Compound-QA task, which concatenates multiple problems into a single prompt to assess both diversity and coherency of reasoning. Our analysis shows that Compound-QA exposes weaknesses in monologue reasoning, evidenced by both quantitative metrics and qualitative reasoning traces. Building on the analysis, we propose a dialogue-based reasoning, named DialogueReason, structured around agents, environment, and interactions. Using PPO with rule-based rewards, we train open-source LLMs (Qwen-QWQ and Qwen-Base) to adopt dialogue reasoning. We evaluate trained models on MATH, AIME, and GPQA datasets, showing that the dialogue reasoning model outperforms monologue models under more complex compound questions. Additionally, we discuss how dialogue-based reasoning helps enhance interpretability, facilitate more intuitive human interaction, and inspire advances in multi-agent system design.

Related papers

• Reasoning or Not? A Comprehensive Evaluation of Reasoning LLMs for Dialogue Summarization [9.994839971567542]
  We present the first comprehensive and systematic evaluation of state-of-the-art reasoning LLMs and non-reasoning LLMs.<n>Contrary to trends in other reasoning-intensive tasks, our findings show that explicit stepwise reasoning does not consistently improve dialogue summarization quality.
  arXiv  Detail & Related papers  (2025-07-02T21:02:41Z)
• Thinking About Thinking: SAGE-nano's Inverse Reasoning for Self-Aware Language Models [0.0]
  Large Language Models (LLMs) have demonstrated remarkable capabilities at solving complex reasoning tasks with Chain-of-Thought prompting.<n>We introduce textbfinverse reasoning, a novel paradigm enabling LLMs to decompose and explain their own reasoning chains post-hoc.<n>Our work creates new avenues for transparent AI systems and closes significant gaps in AI safety, education, and scientific discovery.
  arXiv  Detail & Related papers  (2025-06-30T09:53:41Z)
• Reasoning in Conversation: Solving Subjective Tasks through Dialogue Simulation for Large Language Models [56.93074140619464]
  We propose RiC (Reasoning in Conversation), a method that focuses on solving subjective tasks through dialogue simulation. The motivation of RiC is to mine useful contextual information by simulating dialogues instead of supplying chain-of-thought style rationales. We evaluate both API-based and open-source LLMs including GPT-4, ChatGPT, and OpenChat across twelve tasks.
  arXiv  Detail & Related papers  (2024-02-27T05:37:10Z)
• Dialogue Chain-of-Thought Distillation for Commonsense-aware Conversational Agents [35.6393824052347]
  We propose a framework for dialogue chain-of-thought (CoT) reasoning. We present DOCTOR, a DialOgue Chain-of-ThOught Reasoner. We conduct experiments to show that enhancing dialogue agents with high-quality rationales from DOCTOR significantly improves the quality of their responses.
  arXiv  Detail & Related papers  (2023-10-13T18:17:23Z)
• Towards CausalGPT: A Multi-Agent Approach for Faithful Knowledge Reasoning via Promoting Causal Consistency in LLMs [55.66353783572259]
  Causal-Consistency Chain-of-Thought harnesses multi-agent collaboration to bolster the faithfulness and causality of foundation models.<n>Our framework demonstrates significant superiority over state-of-the-art methods through extensive and comprehensive evaluations.
  arXiv  Detail & Related papers  (2023-08-23T04:59:21Z)
• DiPlomat: A Dialogue Dataset for Situated Pragmatic Reasoning [89.92601337474954]
  Pragmatic reasoning plays a pivotal role in deciphering implicit meanings that frequently arise in real-life conversations. We introduce a novel challenge, DiPlomat, aiming at benchmarking machines' capabilities on pragmatic reasoning and situated conversational understanding.
  arXiv  Detail & Related papers  (2023-06-15T10:41:23Z)
• Prompting and Evaluating Large Language Models for Proactive Dialogues: Clarification, Target-guided, and Non-collaboration [72.04629217161656]
  This work focuses on three aspects of proactive dialogue systems: clarification, target-guided, and non-collaborative dialogues. To trigger the proactivity of LLMs, we propose the Proactive Chain-of-Thought prompting scheme.
  arXiv  Detail & Related papers  (2023-05-23T02:49:35Z)
• FCM: A Fine-grained Comparison Model for Multi-turn Dialogue Reasoning [44.24589471800725]
  This paper proposes a Fine-grained Comparison Model (FCM) to tackle this problem. Inspired by human's behavior in reading comprehension, a comparison mechanism is proposed to focus on the fine-grained differences in the representation of each response candidate.
  arXiv  Detail & Related papers  (2021-09-22T04:16:11Z)
• Dialogue-Based Relation Extraction [53.2896545819799]
  We present the first human-annotated dialogue-based relation extraction (RE) dataset DialogRE. We argue that speaker-related information plays a critical role in the proposed task, based on an analysis of similarities and differences between dialogue-based and traditional RE tasks. Experimental results demonstrate that a speaker-aware extension on the best-performing model leads to gains in both the standard and conversational evaluation settings.
  arXiv  Detail & Related papers  (2020-04-17T03:51:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.