Adversarial Training for Failure-Sensitive User Simulation in Mental Health Dialogue Optimization

• URL: http://arxiv.org/abs/2512.20773v1
• Date: Tue, 23 Dec 2025 21:21:08 GMT
• Title: Adversarial Training for Failure-Sensitive User Simulation in Mental Health Dialogue Optimization
• Abstract summary: A key property of effective simulators is their ability to expose failure modes of the systems they evaluate.<n>We present an adversarial training framework that improves user simulator realism through a competitive dynamic between a generator (user simulator) and a discriminator.
• Score: 1.964225730132177
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Realistic user simulation is crucial for training and evaluating task-oriented dialogue (TOD) systems, yet creating simulators that accurately replicate human behavior remains challenging. A key property of effective simulators is their ability to expose failure modes of the systems they evaluate. We present an adversarial training framework that iteratively improves user simulator realism through a competitive dynamic between a generator (user simulator) and a discriminator. Applied to mental health support chatbots, our approach demonstrates that fine-tuned simulators dramatically outperform zero-shot base models at surfacing system issues, and adversarial training further enhances diversity, distributional alignment, and predictive validity. The resulting simulator achieves a strong correlation between simulated and real failure occurrence rates across diverse chatbot configurations while maintaining low distributional divergence of failure modes. Discriminator accuracy decreases drastically after three adversarial iterations, suggesting improved realism. These results provide evidence that adversarial training is a promising approach for creating realistic user simulators in mental health support TOD domains, enabling rapid, reliable, and cost-effective system evaluation before deployment.

Related papers

• Simulating Students or Sycophantic Problem Solving? On Misconception Faithfulness of LLM Simulators [55.617099475539305]
  Large language models (LLMs) can fluently generate student-like responses, making them attractive as simulated students for training and evaluating AI tutors and human educators.<n>Yet such simulators are typically evaluated by output similarity to real students, not by whether they behave like students with coherent misconceptions during interaction.<n>We introduce a controlled framework for evaluating misconception faithfulness, whether a simulator maintains a misconception-driven belief state and updates selectively when feedback addresses the underlying misconception.
  arXiv  Detail & Related papers  (2026-05-12T20:55:23Z)
• Measuring and Mitigating the Distributional Gap Between Real and Simulated User Behaviors [61.610957638373826]
  We introduce a method to measure the distributional gap between real and simulated user behaviors.<n>Given a dataset of real and simulated conversations, our method extracts representations of user behavior from each conversation.<n>We show that most simulators behave similarly, while a few stand apart.
  arXiv  Detail & Related papers  (2026-05-08T15:09:25Z)
• PSI-Bench: Towards Clinically Grounded and Interpretable Evaluation of Depression Patient Simulators [14.323763649788907]
  We introduce PSI-Bench, an automatic evaluation framework that provides interpretable, clinically grounded diagnostics of depression patient simulator behavior.<n>Using PSI-Bench, we benchmark seven LLMs across two simulator frameworks and find that simulators produce overly long, lexically diverse responses.<n>Results from a human study demonstrate that our benchmark is strongly aligned with expert judgments.
  arXiv  Detail & Related papers  (2026-04-28T16:46:25Z)
• MUSE: Multi-Domain Chinese User Simulation via Self-Evolving Profiles and Rubric-Guided Alignment [15.586402133245313]
  MUSE is a Chinese user simulation framework designed to generate human-like, controllable, and behaviorally consistent responses.<n>We propose Iterative Profile Self-Evolution and Role-Reversal Supervised Fine-Tuning to improve local response realism and human-like expression.<n>Experiments show that MUSE consistently outperforms strong baselines in both utterance-level and session-level evaluations.
  arXiv  Detail & Related papers  (2026-04-15T13:01:00Z)
• Mind the Sim2Real Gap in User Simulation for Agentic Tasks [101.69142591891234]
  We formalize the Sim2Real gap in user simulation and present the first study running the full $$-bench protocol with real humans.<n> Behaviorally, LLM simulators are excessively cooperative, stylistically uniform, and lack realistic frustration or ambiguity.<n>These findings highlight the importance of human validation when using LLM-based user simulators in the agent development cycle.
  arXiv  Detail & Related papers  (2026-03-11T19:12:31Z)
• Sim2Act: Robust Simulation-to-Decision Learning via Adversarial Calibration and Group-Relative Perturbation [54.29523408543184]
  Simulation-to-decision learning enables safe policy training in digital environments without risking real-world deployment.<n>Existing approaches either focus on improving average simulation fidelity or adopt conservative regularization.<n>We propose Sim2Act, a robust simulation-to-decision framework that addresses both simulator and policy robustness.
  arXiv  Detail & Related papers  (2026-03-10T00:51:47Z)
• DiffusionHarmonizer: Bridging Neural Reconstruction and Photorealistic Simulation with Online Diffusion Enhancer [62.18680935878919]
  We introduce DiffusionHarmonizer, an online generative enhancement framework that transforms renderings into temporally consistent outputs.<n>At its core is a single-step temporally-conditioned enhancer capable of running in online simulators on a single GPU.
  arXiv  Detail & Related papers  (2026-02-27T15:35:30Z)
• ARTIS: Agentic Risk-Aware Test-Time Scaling via Iterative Simulation [72.78362530982109]
  ARTIS, Agentic Risk-Aware Test-Time Scaling via Iterative Simulation, is a framework that decouples exploration from commitment.<n>We show that naive LLM-based simulators struggle to capture rare but high-impact failure modes.<n>We introduce a risk-aware tool simulator that emphasizes fidelity on failure-inducing actions.
  arXiv  Detail & Related papers  (2026-02-02T06:33:22Z)
• Autoregressive deep learning for real-time simulation of soft tissue dynamics during virtual neurosurgery [19.464888859778487]
  Traditional numerical solvers often fall short in meeting real-time performance requirements.<n>We introduce a deep learning-based surrogate model that efficiently simulates transient brain deformation.<n>We further integrate the trained surrogate model into an interactive neurosurgical simulation environment, achieving runtimes below 10 ms per simulation step.
  arXiv  Detail & Related papers  (2026-01-20T07:25:32Z)
• PolaRiS: Scalable Real-to-Sim Evaluations for Generalist Robot Policies [88.78188489161028]
  We introduce Policy Evaluation and Environment Reconstruction in Simulation (PolaRiS)<n>PolaRiS is a scalable real-to-sim framework for high-fidelity simulated robot evaluation.<n>We show that PolaRiS evaluations provide a much stronger correlation to real world generalist policy performance than existing simulated benchmarks.
  arXiv  Detail & Related papers  (2025-12-18T18:49:41Z)
• SimBench: Benchmarking the Ability of Large Language Models to Simulate Human Behaviors [58.87134689752605]
  We introduce SimBench, the first large-scale, standardized benchmark for a robust, reproducible science of LLM simulation.<n>We show that even the best LLMs today have limited simulation ability (score: 40.80/100), performance scales log-linearly with model size.<n>We demonstrate that simulation ability correlates most strongly with deep, knowledge-intensive reasoning.
  arXiv  Detail & Related papers  (2025-10-20T13:14:38Z)
• LLM-Powered User Simulator for Recommender System [29.328839982869923]
  We introduce an LLM-powered user simulator to simulate user engagement with items in an explicit manner.<n>Specifically, we identify the explicit logic of user preferences, leverage LLMs to analyze item characteristics and distill user sentiments.<n>We propose an ensemble model that synergizes logical and statistical insights for user interaction simulations.
  arXiv  Detail & Related papers  (2024-12-22T12:00:04Z)
• BeSimulator: A Large Language Model Powered Text-based Behavior Simulator [18.318419980796012]
  We propose BeSimulator as an attempt towards behavior simulation in the context of text-based environments.<n>BeSimulator can generalize across scenarios and achieve long-horizon complex simulation.<n>Our experiments show a significant performance improvement in behavior simulation compared to baselines.
  arXiv  Detail & Related papers  (2024-09-24T08:37:04Z)
• Robust Deep Reinforcement Learning with Adaptive Adversarial Perturbations in Action Space [3.639580365066386]
  We propose an adaptive adversarial coefficient framework to adjust the effect of the adversarial perturbation during training. The appealing feature of our method is that it is simple to deploy in real-world applications and does not require accessing the simulator in advance. The experiments in MuJoCo show that our method can improve the training stability and learn a robust policy when migrated to different test environments.
  arXiv  Detail & Related papers  (2024-05-20T12:31:11Z)
• Choose Your Simulator Wisely: A Review on Open-source Simulators for Autonomous Driving [25.320362844415012]
  There is a growing concern about the validity of algorithms developed and evaluated in simulators. This paper analyzes the evolution of simulators and explains how the functionalities and utilities have developed. Recommendations for select simulators are presented, considering factors such as accessibility, maintenance status, and quality.
  arXiv  Detail & Related papers  (2023-11-18T12:30:41Z)
• Reinforcement Learning with Human Feedback for Realistic Traffic Simulation [53.85002640149283]
  Key element of effective simulation is the incorporation of realistic traffic models that align with human knowledge. This study identifies two main challenges: capturing the nuances of human preferences on realism and the unification of diverse traffic simulation models.
  arXiv  Detail & Related papers  (2023-09-01T19:29:53Z)
• DiAReL: Reinforcement Learning with Disturbance Awareness for Robust Sim2Real Policy Transfer in Robot Control [0.0]
  Delayed Markov decision processes fulfill the Markov property by augmenting the state space of agents with a finite time window of recently committed actions. We introduce a disturbance-augmented Markov decision process in delayed settings as a novel representation to incorporate disturbance estimation in training on-policy reinforcement learning algorithms.
  arXiv  Detail & Related papers  (2023-06-15T10:11:38Z)
• Adversarial learning of neural user simulators for dialogue policy optimisation [14.257597015289512]
  Reinforcement learning based dialogue policies are typically trained in interaction with a user simulator. Current data-driven simulators are trained to accurately model the user behaviour in a dialogue corpus. We propose an alternative method using adversarial learning, with the aim to simulate realistic user behaviour with more variation.
  arXiv  Detail & Related papers  (2023-06-01T16:17:16Z)
• Metaphorical User Simulators for Evaluating Task-oriented Dialogue Systems [80.77917437785773]
  Task-oriented dialogue systems ( TDSs) are assessed mainly in an offline setting or through human evaluation. We propose a metaphorical user simulator for end-to-end TDS evaluation, where we define a simulator to be metaphorical if it simulates user's analogical thinking in interactions with systems. We also propose a tester-based evaluation framework to generate variants, i.e., dialogue systems with different capabilities.
  arXiv  Detail & Related papers  (2022-04-02T05:11:03Z)
• Adversarial Training is Not Ready for Robot Learning [55.493354071227174]
  Adversarial training is an effective method to train deep learning models that are resilient to norm-bounded perturbations. We show theoretically and experimentally that neural controllers obtained via adversarial training are subjected to three types of defects. Our results suggest that adversarial training is not yet ready for robot learning.
  arXiv  Detail & Related papers  (2021-03-15T07:51:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.