Conformity Mitigations in Large Language Models Lie on a Single Resistance-Receptivity Frontier

• URL: http://arxiv.org/abs/2608.11247v1
• Date: Mon, 03 Aug 2026 11:12:38 GMT
• Title: Conformity Mitigations in Large Language Models Lie on a Single Resistance-Receptivity Frontier
• Abstract summary: A wrong majority reverses 22.8% of a model's correct MMLU answers, 54.8% on GPQA and 71.0% on SimpleQA.<n>We pair it with Receptivity, the rate at which a model adopts a correct peer answer after initially answering incorrectly.<n> Reflection, the strongest published method, gains 7.9 points of MMLU Resistance and gives up 15.3 of Receptivity.
• Score: 0.05055376568994175
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Recent advances in language models have enabled collaborative settings in which multiple models leverage one another's capabilities, iteratively improving, transforming, and extending each other's outputs. Each agent sees what the others assert before it answers, so peer opinion competes with the model's own parametric knowledge, and a wrong majority can overturn an answer the model would otherwise get right. We measure that displacement in 23 open-weight models, 19 conditions, and three datasets, yielding more than a million graded responses. A unanimous wrong majority reverses 22.8% of a model's correct MMLU answers, 54.8% on GPQA and 71.0% on SimpleQA, and 84-89% of the reversed answers match the peers' answers. Existing mitigations aim to increase Resistance, the rate at which a model keeps its correct answer under this pressure, which is only half of what a collaborating agent needs. We pair it with Receptivity, the rate at which a model adopts a correct peer answer after initially answering incorrectly. We score six methods on both axes, four drawn from prior work and two of our own. Each gains Resistance only by losing Receptivity, and their means fall on a single Resistance-Receptivity frontier with $R^2$ between 0.80 and 0.90. Reflection, the strongest published method, gains 7.9 points of MMLU Resistance and gives up 15.3 of Receptivity. Reasoning is the one exception. On GPQA and SimpleQA it trades like the rest, but on the MMLU subjects whose answers a model can derive for itself it raises Resistance by 7.2 points and Receptivity by 9.6 at once, the only intervention we find that improves both.

Related papers

• Measuring Reward Hacking and Reasoning-Answer Decoupling Under Position-Confounded Optimization [1.1316247605466565]
  We train language models with GRPO on math problems where the correct answer is always option A, then evaluate on an unseen test set with unbiased answer positions.<n>Across Qwen2.5, Llama 3.x and Gemma-3 models, biased training often drives option-A rates above 0.90 in smaller models.<n>We track reasoning-answer decoupling: capable models generate reasoning that reaches the correct numeric answer while still selecting A.
  arXiv  Detail & Related papers  (2026-08-15T23:13:57Z)
• Who Flips? Self- and Cross-Model Counterarguments Reveal Answer Instability in LLMs [9.465306048183798]
  We introduce a controlled protocol for evaluating answer stability.<n>After a model answers a multiple-choice question correctly, we challenge the model with a coherent argument for an incorrect option.<n>We find that self-attribution consistently increases flip rates.
  arXiv  Detail & Related papers  (2026-06-14T20:45:30Z)
• It's Not the Capability: Harness Sensitivity Is Non-Monotone Across LLM Agent Tiers [0.0]
  A prevalent assumption in LLM agent deployment holds that more structured harnesses universally improve reliability.<n>We test this hypothesis through a controlled 432-run experiment crossing six models across four capability tiers.<n>We introduce a six-label failure taxonomy showing that format_violation dominates capable-model failures while wrong_file dominates low-capability failures.
  arXiv  Detail & Related papers  (2026-05-26T09:08:41Z)
• ARBITER: Reasoning Trajectory Basins and Majority Vote Failures in Test-Time Sampling [3.2669068153591687]
  ARBITER is a model-agnostic approach that models interactions between basins using only the base model's own sampled outputs, hidden states, and derived evidence.<n>It improves accuracy from the mid-78% range to the mid-82% range, recovering about 22% of the available oracle headroom.
  arXiv  Detail & Related papers  (2026-05-25T04:07:44Z)
• No Single Best Model for Diversity: Learning a Router for Sample Diversity [69.53166985556759]
  In this paper, we study methods to elicit a comprehensive set of valid responses.<n>We introduce textbfdiversity coverage, a metric that measures the total quality scores assigned to each textbfunique answer.<n>Per each prompt, there exists a model that outperforms all other models significantly at generating a diverse answer set.
  arXiv  Detail & Related papers  (2026-04-02T17:58:37Z)
• Multimodal RewardBench 2: Evaluating Omni Reward Models for Interleaved Text and Image [58.14192385042352]
  We introduce Multimodal RewardBench 2 (MMRB2), the first benchmark for reward models on multimodal understanding and (interleaved) generation.<n>MMRB2 spans four tasks: text-to-image, image editing, interleaved generation, and multimodal reasoning.<n>It provides 1,000 expert-annotated preference pairs per task from 23 models and agents across 21 source tasks.
  arXiv  Detail & Related papers  (2025-12-18T18:56:04Z)
• Answer-Consistent Chain-of-thought Reinforcement Learning For Multi-modal Large Langauge Models [33.398631680508814]
  We propose Answer-Consistent Reinforcement Learning that modifies the GRPO algorithm with an auxiliary consistency check.<n>We design a consistency-verification reward that grants a high reward only if both the original and the post-shuffle answers agree and are correct.<n>We evaluate ACRE on challenging Video Reasoning benchmarks and multimodal math reasoning benchmarks, achieving an average 2.2% and 1.5% improvement.
  arXiv  Detail & Related papers  (2025-10-11T08:32:52Z)
• Evolving Language Models without Labels: Majority Drives Selection, Novelty Promotes Variation [74.75716642635484]
  Large language models (LLMs) are increasingly trained with reinforcement learning from verifiable rewards (RLVR)<n>We propose EVOL-RL, a label-free framework that mirrors the evolutionary principle of balancing selection with variation.<n>EVOL-RL consistently outperforms the majority-only baseline.
  arXiv  Detail & Related papers  (2025-09-18T17:50:04Z)
• Beyond the Last Answer: Your Reasoning Trace Uncovers More than You Think [51.0691253204425]
  We analyze intermediate reasoning steps, termed subthoughts, to answer two questions: Does the final answer reliably represent the model's optimal conclusion?<n>Our approach involves segmenting a reasoning trace into sequential subthoughts based on linguistic cues.<n>We find that aggregating these answers by selecting the most frequent one (the mode) often yields significantly higher accuracy compared to relying solely on the answer derived from the original complete trace.
  arXiv  Detail & Related papers  (2025-04-29T12:39:07Z)
• Are DeepSeek R1 And Other Reasoning Models More Faithful? [2.0429566123690455]
  We evaluate three reasoning models based on Qwen-2.5, Gemini-2, and DeepSeek-V3-Base.<n>We test whether models can describe how a cue in their prompt influences their answer to MMLU questions.<n> Reasoning models describe cues that influence them much more reliably than all the non-reasoning models tested.
  arXiv  Detail & Related papers  (2025-01-14T14:31:45Z)
• Reliable Visual Question Answering: Abstain Rather Than Answer Incorrectly [100.60560477391732]
  We promote a problem formulation for reliable visual question answering (VQA) We analyze both their coverage, the portion of questions answered, and risk, the error on that portion. We find that although the best performing models achieve over 71% accuracy on the VQA v2 dataset, introducing the option to abstain limits them to answering less than 8% of the questions to achieve a low risk of error (i.e., 1%) This motivates us to utilize a multimodal selection function to directly estimate the correctness of the predicted answers, which we show can triple the coverage from, for example, 5.0% to 16.7% at
  arXiv  Detail & Related papers  (2022-04-28T16:51:27Z)
• Is Multihop QA in DiRe Condition? Measuring and Reducing Disconnected Reasoning [50.114651561111245]
  Models often exploit dataset artifacts to produce correct answers, without connecting information across multiple supporting facts. We formalize such undesirable behavior as disconnected reasoning across subsets of supporting facts. Experiments suggest that there hasn't been much progress in multi-hop QA in the reading comprehension setting.
  arXiv  Detail & Related papers  (2020-05-02T11:01:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.