Can You Break RLVER? Probing Adversarial Robustness of RL-Trained Empathetic Agents

• URL: http://arxiv.org/abs/2605.07138v1
• Date: Fri, 08 May 2026 02:14:12 GMT
• Title: Can You Break RLVER? Probing Adversarial Robustness of RL-Trained Empathetic Agents
• Abstract summary: RLVER has produced language models with strong empathetic performance, evaluated on benchmarks that assume cooperative, honest users.<n>We construct the Adversarial Empathy Benchmark AEB and introduce the Emotional Consistency Score ECS to evaluate empathetic robustness under adversarial conditions.<n>We interpret the ECS--FS (Final Score) gap as a behavioral/legibility dissociation inside this simulator family, not as evidence about internal understanding or clinical readiness.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Reinforcement learning from verifiable emotion rewards RLVER has produced language models with strong empathetic performance, evaluated on benchmarks that assume cooperative, honest users. Yet real emotional interactions systematically violate this assumption: users gaslight, escalate, and pressure AI systems for unconditional validation, dynamics that cooperative benchmarks cannot surface. We construct the Adversarial Empathy Benchmark AEB and introduce the Emotional Consistency Score ECS to evaluate empathetic robustness under adversarial conditions. AEB comprises six psychologically grounded adversarial trajectory types with discriminative reward structures that penalize formulaic responses; ECS formally disentangles a model's capacity to track user emotional states from its capacity to improve them. In a controlled experiment across eight scenario-matched conditions (think and no-think conditions on 2 RLVER models, and 2 base models (Qwen 1.5B and 7B) with 480 adversarial dialogues), RLVER-PPO-Think substantially outperforms the same-scale untuned baseline (0.963 vs. 0.761, \(p<0.001, r=0.688\)), with zero dialogue collapses and 47\% higher hidden-intention detection. However, ECS remains nearly flat and is not significantly different for RLVER-PPO-Think versus Base-7B-Think (\(p=0.650\)): RL training improves emotional responsiveness without measurable gains in observable state tracking. We interpret the ECS--FS (Final Score) gap as a behavioral/legibility dissociation inside this simulator family, not as evidence about internal understanding or clinical readiness.

Related papers

• AffectOmni: RL-Verifiable People-Centric Grounded Affective Reasoning for Social and Art-Related Scenes [59.134503227559634]
  Multimodal large language models (MLLMs) achieve strong performance on VQA and scene understanding.<n>Models may predict correct answers while neglecting people-centric cues such as micro expressions and body language.<n>We propose Affect Omni, a framework for verifiable affective reasoning.
  arXiv  Detail & Related papers  (2026-08-24T11:25:04Z)
• EmoS: A Theory-Grounded Framework for Evaluating and Aligning Emotional Intelligence in Spoken Language Models [66.28519000191191]
  EmoSBench is the first comprehensive EI evaluation benchmark for Spoken Language Models.<n>EmoS reaches 83.8% accuracy, approaching human-level performance.
  arXiv  Detail & Related papers  (2026-08-10T06:58:30Z)
• CompanionBench: A Theory-Anchored, Real-World-Grounded Benchmark for AI Emotional Companionship [7.643253506288442]
  Existing benchmarks use hand-authored scenarios and prompted simulators.<n>We introduce CompanionBench, an interactive bilingual benchmark.<n>It is the first companion benchmark to ground both its scenarios and a trained user simulator in real-world data.
  arXiv  Detail & Related papers  (2026-08-03T10:44:04Z)
• EIBench: A Simulator-Based Benchmark and Turn-Credit RL for Emotion Management [78.71074071187753]
  We introduce EIBench, a simulator-based benchmark for interactive emotion management.<n> EIBench contains 2,222 scenarios, with 2,009 for training and 213 for held-out testing.<n>Current models perform well on support and rapport-building scenes, but struggle with boundary maintenance under user pressure.
  arXiv  Detail & Related papers  (2026-06-14T01:22:03Z)
• Enhancing LLM Metacognition via Cognitive Pairwise Training [56.19495984013656]
  Reinforcement learning with verifiable rewards can make models more willing to give confident answers when evidence or reasoning is unreliable.<n>Existing SFT or RL methods mainly teach LLMs to refuse or express uncertainty at the response level.<n>We propose Cognitive Pairwise Training (CPT), a cognitive mid-training alignment stage that turns pairwise comparisons over reasoning traces into a reusable alignment signal.
  arXiv  Detail & Related papers  (2026-05-30T19:53:19Z)
• Simulating Students or Sycophantic Problem Solving? On Misconception Faithfulness of LLM Simulators [55.617099475539305]
  Large language models (LLMs) can fluently generate student-like responses, making them attractive as simulated students for training and evaluating AI tutors and human educators.<n>Yet such simulators are typically evaluated by output similarity to real students, not by whether they behave like students with coherent misconceptions during interaction.<n>We introduce a controlled framework for evaluating misconception faithfulness, whether a simulator maintains a misconception-driven belief state and updates selectively when feedback addresses the underlying misconception.
  arXiv  Detail & Related papers  (2026-05-12T20:55:23Z)
• FreakOut-LLM: The Effect of Emotional Stimuli on Safety Alignment [13.02804082409836]
  Safety-aligned LLMs go through refusal training to reject harmful requests, but whether these mechanisms remain effective under emotionally charged stimuli is unexplored.<n>We introduce FreakOut-LLM, a framework investigating whether emotional context compromises safety alignment in adversarial settings.
  arXiv  Detail & Related papers  (2026-04-05T13:37:52Z)
• Dynasto: Validity-Aware Dynamic-Static Parameter Optimization for Autonomous Driving Testing [6.969678532857503]
  We propose Dynasto, a two-step testing approach to uncover safety-critical failures in autonomous driving systems.<n>First, we train an adversarial agent using reinforcement learning with temporal-logic-based validity criteria.<n>Second, a genetic algorithm searches over initial conditions while replaying the adversary's failure-inducing behaviors to reveal additional failures that the RL agent alone does not uncover.
  arXiv  Detail & Related papers  (2026-03-22T22:35:30Z)
• RetroAgent: From Solving to Evolving via Retrospective Dual Intrinsic Feedback [54.39884046754265]
  RetroAgent is an online RL framework that enables agents to master complex interactive environments.<n>Our experiments show that RetroAgent achieves state-of-the-art (SOTA) performance.
  arXiv  Detail & Related papers  (2026-03-09T16:23:33Z)
• Cognitive Load Limits in Large Language Models: Benchmarking Multi-Hop Reasoning [0.0]
  Large Language Models (LLMs) excel at isolated tasks, but their reasoning under cognitive load remains poorly understood.<n>We introduce a formal theory of computational cognitive load, positing that extraneous, task-irrelevant information (Context Saturation) and interference from task-switching are key mechanisms that degrade performance.
  arXiv  Detail & Related papers  (2025-09-23T19:36:56Z)
• OmniEAR: Benchmarking Agent Reasoning in Embodied Tasks [52.87238755666243]
  We present OmniEAR, a framework for evaluating how language models reason about physical interactions, tool usage, and multi-agent coordination in embodied tasks.<n>We model continuous physical properties and complex spatial relationships across 1,500 scenarios spanning household and industrial domains.<n>Our systematic evaluation reveals severe performance degradation when models must reason from constraints.
  arXiv  Detail & Related papers  (2025-08-07T17:54:15Z)
• RLVER: Reinforcement Learning with Verifiable Emotion Rewards for Empathetic Agents [67.46032287312339]
  Large language models (LLMs) excel at logical and algorithmic reasoning, yet their emotional intelligence (EQ) still lags far behind their cognitive prowess.<n>We introduce RLVER, the first end-to-end reinforcement learning framework that leverages verifiable emotion rewards from simulated users.<n>Our results show that RLVER is a practical route toward emotionally intelligent and broadly capable language agents.
  arXiv  Detail & Related papers  (2025-07-03T18:33:18Z)
• Sentiment Simulation using Generative AI Agents [0.0]
  We present a framework for sentiment simulation using generative AI agents embedded with psychologically rich profiles.<n>Agents are instantiated from a nationally representative survey of 2,485 Filipino respondents.<n>Our findings establish a scalable framework for sentiment modeling through psychographically grounded AI agents.
  arXiv  Detail & Related papers  (2025-05-28T08:50:56Z)
• GTR: Guided Thought Reinforcement Prevents Thought Collapse in RL-based VLM Agent Training [62.536191233049614]
  Reinforcement learning with verifiable outcome rewards (RLVR) has effectively scaled up chain-of-thought (CoT) reasoning in large language models (LLMs)<n>This work investigates this problem through extensive experiments on complex card games, such as 24 points, and embodied tasks from ALFWorld.<n>We find that when rewards are based solely on action outcomes, RL fails to incentivize CoT reasoning in VLMs, instead leading to a phenomenon we termed thought collapse.
  arXiv  Detail & Related papers  (2025-03-11T15:17:02Z)
• Towards a Competitive End-to-End Speech Recognition for CHiME-6 Dinner Party Transcription [73.66530509749305]
  In this paper, we argue that, even in difficult cases, some end-to-end approaches show performance close to the hybrid baseline. We experimentally compare and analyze CTC-Attention versus RNN-Transducer approaches along with RNN versus Transformer architectures. Our best end-to-end model based on RNN-Transducer, together with improved beam search, reaches quality by only 3.8% WER abs. worse than the LF-MMI TDNN-F CHiME-6 Challenge baseline.
  arXiv  Detail & Related papers  (2020-04-22T19:08:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.