When Specifications Conflict: A Symmetry-Based Framework for Measuring LLM Preferences

• URL: http://arxiv.org/abs/2607.28384v1
• Date: Thu, 30 Jul 2026 15:42:32 GMT
• Title: When Specifications Conflict: A Symmetry-Based Framework for Measuring LLM Preferences
• Abstract summary: Large language models (LLMs) are increasingly required to integrate multiple sources of information.<n>We propose a controlled experimental framework for studying model preferences under conflicting specifications.
• Score: 3.046191671479774
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) are increasingly required to integrate multiple sources of information that may be inconsistent or conflicting. However, there is still a lack of controllable and attributable methods for analyzing how models resolve conflicts between competing specifications. We propose a controlled experimental framework for studying model preferences under conflicting specifications. By constructing specifications with explicit conflicts, the framework enables model choices between competing specifications to be directly observed and analyzed. A symmetry-based design further reduces confounding factors, allowing preferences across representation types to be compared systematically. We evaluate the framework on an executable mathematical benchmark with 550 conflict instances spanning 11 function families, comparing four representation types: pure natural language, formal language, naturalized formal language, and input--output examples. Results show systematic preference patterns rather than random behavior, with a consistent ordering: $ \text{Formal} \approx \text{Naturalized Formal} > \text{Pure Natural Language} > \text{Input--Output Examples} $. Example effects further depend on model capability and function family. We extend the framework to heterogeneous specification conflicts in Boolean algebra, code generation, and the clinical domain, demonstrating its applicability across diverse tasks and specification forms. The framework provides a unified approach for measuring how LLMs resolve conflicts between competing sources of information.

Related papers

• PRIME: Evaluating Prompt Resolution Under Incompatible Instructions in LLMs [1.7243216387069673]
  Large language models (LLMs) often encounter conflicting prompts.<n>Current instruction following benchmarks assess those meta-instructions in isolation.<n>We introduce a framework textitPRIME to analyze behavior of LLMs when provided with conflicting instructions.
  arXiv  Detail & Related papers  (2026-06-21T12:30:23Z)
• CORE: Conflict-Oriented Reasoning for General Multimodal Manipulation Detection [56.64398465636452]
  generative AI has made multimodal fake news increasingly realistic and pervasive, posing severe threats to public trust and social stability.<n>We propose textbfConflict-textbfOriented textbfREasoning (textbfCORE) framework, an effective paradigm that learns to endows multimodal large language models (MLLMs) with explicit conflict-capturing capability.<n> CORE achieves robust and generalizable conflict detection, effectively and rapidly adapting to unseen manipulation types with a few samples or in even zero-shot settings.
  arXiv  Detail & Related papers  (2026-06-02T02:53:48Z)
• ABLE: Representing and Mapping LLMs via Attribution-Based Large-model Embedding [23.27523995613204]
  We propose ABLE (Attribution-Based Large-model Embedding), a framework that leverages the interpretability space to construct model representations.<n>By aggregating gradient-based feature attributions via a tokenizer-agnostic word-level alignment, ABLE captures model-specific input-sensitivity patterns.<n>Our training-free approach achieves competitive or superior performance in relation prediction, model routing, and benchmark score prediction.
  arXiv  Detail & Related papers  (2026-04-20T04:55:11Z)
• Gaussian Joint Embeddings For Self-Supervised Representation Learning [0.0]
  Self-parametric representation learning often relies on deterministic predictive architectures to align context and target views in latent space.<n>We propose a probabilistic alternative based on generative joint modeling.
  arXiv  Detail & Related papers  (2026-03-26T00:54:54Z)
• Stress-Testing Model Specs Reveals Character Differences among Language Models [23.505192393830807]
  Large language models (LLMs) are increasingly trained from AI constitutions and model specifications.<n>We present a systematic methodology for stress-testing model character specifications.<n>We identify numerous cases of principle contradictions and interpretive ambiguities in current model specs.
  arXiv  Detail & Related papers  (2025-10-09T02:24:37Z)
• ECon: On the Detection and Resolution of Evidence Conflicts [56.89209046429291]
  The rise of large language models (LLMs) has significantly influenced the quality of information in decision-making systems. This study introduces a method for generating diverse, validated evidence conflicts to simulate real-world misinformation scenarios.
  arXiv  Detail & Related papers  (2024-10-05T07:41:17Z)
• AdaCAD: Adaptively Decoding to Balance Conflicts between Contextual and Parametric Knowledge [57.66282463340297]
  Knowledge conflict arises from discrepancies between information in the context of a large language model and the knowledge stored in its parameters.<n>We propose a fine-grained, instance-level approach called AdaCAD, which dynamically infers the weight of adjustment based on the degree of conflict.<n>We show that ADACAD consistently outperforms other decoding baselines with average QA accuracy gains of 14.21% (absolute) over a static contrastive baseline, and improves the factuality of summaries by 6.19 (AlignScore)
  arXiv  Detail & Related papers  (2024-09-11T16:35:18Z)
• Shape Arithmetic Expressions: Advancing Scientific Discovery Beyond Closed-Form Equations [56.78271181959529]
  Generalized Additive Models (GAMs) can capture non-linear relationships between variables and targets, but they cannot capture intricate feature interactions. We propose Shape Expressions Arithmetic ( SHAREs) that fuses GAM's flexible shape functions with the complex feature interactions found in mathematical expressions. We also design a set of rules for constructing SHAREs that guarantee transparency of the found expressions beyond the standard constraints.
  arXiv  Detail & Related papers  (2024-04-15T13:44:01Z)
• Representation Disentaglement via Regularization by Causal Identification [3.9160947065896803]
  We propose the use of a causal collider structured model to describe the underlying data generative process assumptions in disentangled representation learning. For this, we propose regularization by identification (ReI), a modular regularization engine designed to align the behavior of large scale generative models with the disentanglement constraints imposed by causal identification.
  arXiv  Detail & Related papers  (2023-02-28T23:18:54Z)
• Variable Importance Matching for Causal Inference [73.25504313552516]
  We describe a general framework called Model-to-Match that achieves these goals. Model-to-Match uses variable importance measurements to construct a distance metric. We operationalize the Model-to-Match framework with LASSO.
  arXiv  Detail & Related papers  (2023-02-23T00:43:03Z)
• Switchable Representation Learning Framework with Self-compatibility [50.48336074436792]
  We propose a Switchable representation learning Framework with Self-Compatibility (SFSC) SFSC generates a series of compatible sub-models with different capacities through one training process. SFSC achieves state-of-the-art performance on the evaluated datasets.
  arXiv  Detail & Related papers  (2022-06-16T16:46:32Z)
• Automatically detecting the conflicts between software requirements based on finer semantic analysis [8.035669538927623]
  We propose an approach Finer Semantic Analysis-based Requirements Conflict Detector (FSARC) to automatically detecting the conflicts between the given natural language functional requirements. We build a harmonized semantic meta-model of functional requirements with the form of eight-tuple. And we define seven types of conflicts as long as their detecting rules on the ground of their text pattern and semantical dependency.
  arXiv  Detail & Related papers  (2021-03-03T08:39:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.