ESCRAG-R1: Retrieval-Augmented Reinforcement Learning for Emotional Support Conversation

• URL: http://arxiv.org/abs/2608.21925v1
• Date: Sat, 22 Aug 2026 11:32:22 GMT
• Title: ESCRAG-R1: Retrieval-Augmented Reinforcement Learning for Emotional Support Conversation
• Abstract summary: Emotional Support Conversation systems aim to provide holistic support by balancing professional therapeutic competence with natural empathy.<n>Existing methods struggle to simultaneously achieve structured, stage-aware reasoning and seamless empathy-expertise alignment.<n>We propose ESCRAG-R1, a unified framework that integrates retrieval-based psychological guidance into Group Relative Policy Optimization.
• Score: 11.720936066131893
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Emotional Support Conversation (ESC) systems aim to provide holistic support by balancing professional therapeutic competence with natural empathy. However, existing methods struggle to simultaneously achieve structured, stage-aware reasoning and seamless empathy-expertise alignment, often resulting in an artificial splicing of clinical strategies and generic reassurance. To overcome these limitations, we propose ESCRAG-R1, a unified framework that integrates retrieval-based psychological guidance into Group Relative Policy Optimization (GRPO). By incorporating retrieval into the reinforcement learning loop, ESCRAG-R1 transforms external knowledge into a robust learning signal that stimulates explicit internal reasoning prior to generation and fundamentally reshapes the model's internal policy. To provide the reliable supervision required for this optimization, we construct ESC-Preference, a high-quality dataset based on a Client--Counselor--Judge evaluation framework that delivers precise, empathy-aware reward signals. Extensive experiments demonstrate that ESCRAG-R1 significantly outperforms existing baselines by mitigating superficial splicing and realizing a natural integration of professional guidance and empathetic expression. Code and datasets are released at https://github.com/Matcha-Liu/ESCRAG-R1.

Related papers

• DeepSAGE: Stage-Aware Reinforcement Learning for Structured CBT Counseling Dialogue [44.08107411670182]
  We present DeepSAGE, a framework for stage-aware counseling dialogue grounded in Cognitive Behavioral Therapy (CBT)<n>DeepSAGE represents the session as eleven stages with explicit therapeutic objectives, with an external controller determines stage completion.<n>DeepSAGE elicits higher simulated client engagement and openness and achieves the strongest balance of stage-goal completion and dialogue efficiency.
  arXiv  Detail & Related papers  (2026-08-23T21:40:58Z)
• Reputation-driven Cooperation in Lattice-based Decentralized Federated Learning through Evolutionary Game Theory [0.3572141827285953]
  This paper proposes a novel EGT framework designed to analyze strategic evolution and enhance overall system performance.<n>We model peer-to-peer (P2P) interactions on a lattice network structure under the assumption of bounded rationality.<n>We integrate a reputation-based reward-and-punishment mechanism to effectively deter free-riding behaviors.
  arXiv  Detail & Related papers  (2026-08-02T12:23:45Z)
• PDCR: Perception-Decomposed Confidence Reward for Vision-Language Reasoning [80.94559742826083]
  Reinforcement Learning with Verifiable Rewards (RLVR) traditionally relies on a sparse, outcome-based signal.<n>Recent work shows that providing a fine-grained, model-intrinsic signal effectively improves language reasoning training by providing step-level guidance without costly external models.<n>While effective for unimodal text, we find that naively applying this global reward to vision-language (V-L) reasoning is a suboptimal strategy.<n>We propose Perception-Decomposed Confidence Reward (PDCR), a framework that solves this by aligning the reward structure with the task's heterogeneous nature.
  arXiv  Detail & Related papers  (2026-05-13T12:55:18Z)
• Chunking, Retrieval, and Re-ranking: An Empirical Evaluation of RAG Architectures for Policy Document Question Answering [0.0]
  The integration of Large Language Models (LLMs) into the public health policy sector offers a transformative approach to navigating the vast repositories of regulatory guidance maintained by agencies such as the Centers for Disease Control and Prevention (CDC)<n>The propensity for LLMs to generate hallucinations, defined as plausible but factually incorrect assertions, presents a critical barrier to the adoption of these technologies in high-stakes environments where information integrity is non-negotiable.<n>This empirical evaluation explores the effectiveness of Retrieval-Augmented Generation (RAG) architectures in mitigating these risks by grounding generative outputs in authoritative document context.
  arXiv  Detail & Related papers  (2026-01-21T20:52:48Z)
• Character-R1: Enhancing Role-Aware Reasoning in Role-Playing Agents via RLVR [67.66592867046229]
  Character-R1 is a framework designed to provide verifiable reward signals for effective role-aware reasoning.<n>Our framework comprises three core designs: Cognitive Focus Reward, Reference-Guided Reward and Character-Conditioned Reward Normalization.
  arXiv  Detail & Related papers  (2026-01-08T05:33:37Z)
• LLM-Assisted AHP for Explainable Cyber Range Evaluation [0.0]
  Cyber Ranges (CRs) have emerged as prominent platforms for cybersecurity training and education.<n>This paper proposes an evaluation framework for CRs, emphasizing mission-critical settings.
  arXiv  Detail & Related papers  (2025-12-11T10:07:15Z)
• Unlocking the Essence of Beauty: Advanced Aesthetic Reasoning with Relative-Absolute Policy Optimization [63.169050703903515]
  We propose Aes-R1, a comprehensive aesthetic reasoning framework with reinforcement learning (RL)<n>Aes-R1 integrates a pipeline, AesCoT, to construct and filter high-quality chain-of-thought aesthetic reasoning data.<n>Experiments demonstrate that Aes-R1 improves the backbone's average PLCC/SRCC by 47.9%/34.8%.
  arXiv  Detail & Related papers  (2025-09-26T04:55:00Z)
• Affordance-R1: Reinforcement Learning for Generalizable Affordance Reasoning in Multimodal Large Language Model [23.56313087226691]
  Affordance grounding focuses on predicting the specific regions of objects that are associated with the actions to be performed by robots.<n>Existing models often neglect the affordance shared among different objects because they lack the Chain-of-Thought(CoT) reasoning abilities.<n>We propose Affordance-R1, the first unified affordance grounding framework that integrates cognitive CoT guided Group Relative Policy Optimization.
  arXiv  Detail & Related papers  (2025-08-08T10:39:04Z)
• SEEA-R1: Tree-Structured Reinforcement Fine-Tuning for Self-Evolving Embodied Agents [58.174206358223415]
  Self-Evolving Embodied Agents-R1, or SEEA-R1, is the first reinforcement fine-tuning framework designed for self-evolving embodied agents.<n>We show that SEEA-R1 can support autonomous adaptation and reward-driven self-evolution.
  arXiv  Detail & Related papers  (2025-06-26T18:00:07Z)
• Ψ-Arena: Interactive Assessment and Optimization of LLM-based Psychological Counselors with Tripartite Feedback [51.26493826461026]
  We propose Psi-Arena, an interactive framework for comprehensive assessment and optimization of large language models (LLMs)<n>Arena features realistic arena interactions that simulate real-world counseling through multi-stage dialogues with psychologically profiled NPC clients.<n>Experiments across eight state-of-the-art LLMs show significant performance variations in different real-world scenarios and evaluation perspectives.
  arXiv  Detail & Related papers  (2025-05-06T08:22:51Z)
• A Regularized Implicit Policy for Offline Reinforcement Learning [54.7427227775581]
  offline reinforcement learning enables learning from a fixed dataset, without further interactions with the environment. We propose a framework that supports learning a flexible yet well-regularized fully-implicit policy. Experiments and ablation study on the D4RL dataset validate our framework and the effectiveness of our algorithmic designs.
  arXiv  Detail & Related papers  (2022-02-19T20:22:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.