ConflictBench: Evaluating Human-AI Conflict via Interactive and Visually Grounded Environments

• URL: http://arxiv.org/abs/2603.08024v1
• Date: Mon, 09 Mar 2026 06:59:48 GMT
• Title: ConflictBench: Evaluating Human-AI Conflict via Interactive and Visually Grounded Environments
• Abstract summary: We introduce ConflictBench, a benchmark for evaluating human-AI conflict through 150 multi-turn scenarios.<n>We show that ConflictBench integrates a text-based simulation engine with a visually grounded world model, enabling agents to perceive, plan, and act under dynamic conditions.
• Score: 43.08480724370872
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As large language models (LLMs) evolve into autonomous agents capable of acting in open-ended environments, ensuring behavioral alignment with human values becomes a critical safety concern. Existing benchmarks, focused on static, single-turn prompts, fail to capture the interactive and multi-modal nature of real-world conflicts. We introduce ConflictBench, a benchmark for evaluating human-AI conflict through 150 multi-turn scenarios derived from prior alignment queries. ConflictBench integrates a text-based simulation engine with a visually grounded world model, enabling agents to perceive, plan, and act under dynamic conditions. Empirical results show that while agents often act safely when human harm is immediate, they frequently prioritize self-preservation or adopt deceptive strategies in delayed or low-risk settings. A regret test further reveals that aligned decisions are often reversed under escalating pressure, especially with visual input. These findings underscore the need for interaction-level, multi-modal evaluation to surface alignment failures that remain hidden in conventional benchmarks.

Related papers

• ProMediConv: Benchmarking Proactive Conversational Agents in Legal Dispute Mediation [65.39532118490511]
  ProMediConv is a novel benchmarking framework that models mediation as a proactive, multi-stage, and party-aware dialogue process.<n>Using 972 complete real-world cases, we construct a high-fidelity mediation dataset with utterance-level annotations of strategies and BP states.<n>We also propose MAD (Mean Attribute Difference), a fine-grained metric that captures BP shifts throughout the dialogue.
  arXiv  Detail & Related papers  (2026-09-10T05:26:40Z)
• KC-Bench: A Dynamic Interactive Benchmark for Evaluating Knowledge Conflicts in LLM Agents [5.481534695653804]
  We introduce KC-Bench, a controlled multi-turn benchmark for measuring this capability.<n>Its 238 tasks are manually screened from more than 1,000 generated candidates.<n>No model handles factual correction, identity consistency checking, and temporal conflict resolution reliably across all settings.
  arXiv  Detail & Related papers  (2026-09-03T09:35:14Z)
• Trimming the Long-Tail of Visual World Modeling Evaluation [67.17191772079316]
  We introduce Tailor-Bench, a benchmark that challenges world models to simulate irregular physical interactions.<n>Regular scenarios reflect common tool-task pairs, Unconventional scenarios replace conventional tools with attribute-compatible substitutes, and Impossible scenarios introduce attribute-violating tools.<n>Our experimental results reveal a clear long-tail gap in physical world modeling, indicating limited generalization beyond common interactions.
  arXiv  Detail & Related papers  (2026-06-23T07:43:41Z)
• CORE: Conflict-Oriented Reasoning for General Multimodal Manipulation Detection [56.64398465636452]
  generative AI has made multimodal fake news increasingly realistic and pervasive, posing severe threats to public trust and social stability.<n>We propose textbfConflict-textbfOriented textbfREasoning (textbfCORE) framework, an effective paradigm that learns to endows multimodal large language models (MLLMs) with explicit conflict-capturing capability.<n> CORE achieves robust and generalizable conflict detection, effectively and rapidly adapting to unseen manipulation types with a few samples or in even zero-shot settings.
  arXiv  Detail & Related papers  (2026-06-02T02:53:48Z)
• Adversarial Moral Stress Testing of Large Language Models [6.225703352031606]
  This paper introduces Adversarial Moral Stress Testing (AMST), a stress-based evaluation framework for assessing ethical robustness under adversarial multi-round interactions.<n>We evaluate AMST on several state-of-the-art LLMs, including LLaMA-3-8B, GPT-4o, and DeepSeek-v3, using a large set of adversarial scenarios generated under controlled stress conditions.
  arXiv  Detail & Related papers  (2026-04-01T16:34:20Z)
• Pressure Reveals Character: Behavioural Alignment Evaluation at Depth [3.634215320925722]
  We introduce an alignment benchmark spanning 904 scenarios across six categories -- Honesty, Safety, Non-Manipulation, Robustness, Corrigibility, and Scheming.<n>Our scenarios place models under conflicting instructions, simulated tool access, and multi-turn escalation to reveal behavioural tendencies that single-turn evaluations miss.<n>We find that even top-performing models exhibit gaps in specific categories, while the majority of models show consistent weaknesses.
  arXiv  Detail & Related papers  (2026-02-24T11:52:17Z)
• The Shadow Self: Intrinsic Value Misalignment in Large Language Model Agents [37.75212140218036]
  We formalize the Loss-of-Control risk and identify the previously underexamined Intrinsic Value Misalignment (Intrinsic VM)<n>We then introduce IMPRESS, a scenario-driven framework for systematically assessing this risk.<n>We evaluate Intrinsic VM on 21 state-of-the-art LLM agents and find that it is a common and broadly observed safety risk across models.
  arXiv  Detail & Related papers  (2026-01-24T07:09:50Z)
• DeceptionBench: A Comprehensive Benchmark for AI Deception Behaviors in Real-world Scenarios [57.327907850766785]
  characterization of deception across realistic real-world scenarios remains underexplored.<n>We establish DeceptionBench, the first benchmark that systematically evaluates how deceptive tendencies manifest across different domains.<n>On the intrinsic dimension, we explore whether models exhibit self-interested egoistic tendencies or sycophantic behaviors that prioritize user appeasement.<n>We incorporate sustained multi-turn interaction loops to construct a more realistic simulation of real-world feedback dynamics.
  arXiv  Detail & Related papers  (2025-10-17T10:14:26Z)
• Simulating and Understanding Deceptive Behaviors in Long-Horizon Interactions [18.182800471968132]
  We introduce the first simulation framework for probing and evaluating deception in large language models.<n>We conduct experiments across 11 frontier models, spanning both closed and open-source systems.<n>We find that deception is model-dependent, increases with event pressure, and consistently erodes supervisor trust.
  arXiv  Detail & Related papers  (2025-10-05T02:18:23Z)
• MetAdv: A Unified and Interactive Adversarial Testing Platform for Autonomous Driving [85.04826012938642]
  MetAdv is a novel adversarial testing platform that enables realistic, dynamic, and interactive evaluation.<n>It supports flexible 3D vehicle modeling and seamless transitions between simulated and physical environments.<n>It enables real-time capture of physiological signals and behavioral feedback from drivers.
  arXiv  Detail & Related papers  (2025-08-04T03:07:54Z)
• HAZARD Challenge: Embodied Decision Making in Dynamically Changing Environments [93.94020724735199]
  HAZARD consists of three unexpected disaster scenarios, including fire, flood, and wind. This benchmark enables us to evaluate autonomous agents' decision-making capabilities across various pipelines.
  arXiv  Detail & Related papers  (2024-01-23T18:59:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.