Uncertainty, bias and the institution bootstrapping problem
- URL: http://arxiv.org/abs/2504.21579v1
- Date: Wed, 30 Apr 2025 12:36:06 GMT
- Title: Uncertainty, bias and the institution bootstrapping problem
- Abstract summary: We propose that misperception, specifically, agents' erroneous belief that an institution already exists, could resolve this paradox.<n>We show how these factors collectively mitigate the bootstrapping problem.<n>Our analysis underscores the importance of incorporating human-like cognitive constraints, not just idealized rationality, into models of institutional emergence and resilience.
- Score: 0.0
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Institutions play a critical role in enabling communities to manage common-pool resources and avert tragedies of the commons. However, a fundamental issue arises: Individuals typically perceive participation as advantageous only after an institution is established, creating a paradox: How can institutions form if no one will join before a critical mass exists? We term this conundrum the institution bootstrapping problem and propose that misperception, specifically, agents' erroneous belief that an institution already exists, could resolve this paradox. By integrating well-documented psychological phenomena, including cognitive biases, probability distortion, and perceptual noise, into a game-theoretic framework, we demonstrate how these factors collectively mitigate the bootstrapping problem. Notably, unbiased perceptual noise (e.g., noise arising from agents' heterogeneous physical or social contexts) drastically reduces the critical mass of cooperators required for institutional emergence. This effect intensifies with greater diversity of perceptions. We explain this counter-intuitive result through asymmetric boundary conditions: proportional underestimation of low-probability sanctions produces distinct outcomes compared to equivalent overestimation. Furthermore, the type of perceptual distortion, proportional versus absolute, yields qualitatively different evolutionary pathways. These findings challenge conventional assumptions about rationality in institutional design, highlighting how "noisy" cognition can paradoxically enhance cooperation. Finally, we contextualize these insights within broader discussions of multi-agent system design and collective action. Our analysis underscores the importance of incorporating human-like cognitive constraints, not just idealized rationality, into models of institutional emergence and resilience.
Related papers
- The Epistemic Politics of AI Anthropomorphism [0.0]
The paper argues that the dominant anthropomorphism frame operates from a position of institutional advantage.<n>It argues that the framing does not simply manage risk. It adjudicates the legitimacy of human experience in interaction with a phenomenon whose nature the field itself has not resolved.
arXiv Detail & Related papers (2026-08-02T03:20:50Z) - To Compare, or Not to Compare: On Methodological Practices in Evaluating Social Bias [61.40435736418359]
We introduce a unified framework that standardizes heterogeneous benchmarks to contrast isolated demographic assessments with forced-choice comparative settings.<n>We show that comparative settings act as aggressive catalysts for latent discrimination, a shift primarily driven by underspecified contexts.<n>Finally, we demonstrate that this comparative prejudice is a generalized phenomenon that scales positively with model size.
arXiv Detail & Related papers (2026-06-23T13:53:50Z) - The Crowded Embedding Space: A Mean-Field Mechanism for Emergent Marginalization in Retrieval-Augmented Agents [51.56484100374058]
We show that the high document density required to serve majority interests can geometrically overcrowd the retrieval neighborhood of a semantically similar minority.<n>We introduce a formal framework to analyze how such goal collisions in dense retrieval induce fundamental performance limits.<n>We prove that such objectives drive the system to self-organize into a state that exclusively serves majority interests.
arXiv Detail & Related papers (2026-06-01T20:40:30Z) - Provably Secure Agent Guardrail [89.79561918065122]
Existing defense architectures rely on empirical semantic guardrails and probabilistic large model adjudicators.<n>This paper proposes a new security paradigm for agents based on the fundamental limitations of logical reasoning.
arXiv Detail & Related papers (2026-05-28T02:12:41Z) - Taming Actor-Observer Asymmetry in Agents via Dialectical Alignment [59.536125286960186]
Multi-agent frameworks assigning specialized roles are increasingly adopted to enable self-reflection and mutual auditing.<n>We find it simultaneously induces a human-like cognitive bias known as Actor-Observer Asymmetry.<n>ReTAS guides agents to synthesize conflicting viewpoints into an objective consensus.
arXiv Detail & Related papers (2026-04-21T15:05:58Z) - A Survey of Reasoning in Autonomous Driving Systems: Open Challenges and Emerging Paradigms [49.66022971508878]
We argue that reasoning should be elevated from a modular component to the system's cognitive core.<n>We derive and systematize seven core reasoning challenges, such as the responsiveness-reasoning trade-off and social-game reasoning.<n>We identify a fundamental and unresolved tension between the high-latency, deliberative nature of LLM-based reasoning and the millisecond-scale, safety-critical demands of vehicle control.
arXiv Detail & Related papers (2026-03-11T07:40:53Z) - Stability of AI Governance Systems: A Coupled Dynamics Model of Public Trust and Social Disruptions [0.49157446832511503]
This paper proposes a rigorous coupled dynamics model that integrates a discrete-time Hawkes process.<n>A key innovation is the bidirectional feedback mechanism: declining trust amplifies the intensity of subsequent controversy events.<n> Numerical experiments reveal how echo chamber network structures and media amplification accelerate governance failure.
arXiv Detail & Related papers (2026-03-10T12:24:30Z) - Policy myopia as a mechanism of gradual disempowerment in Post-AGI governance, Circa 2049 [0.0]
Post-AGI information systems will transform how institutions make decisions in ways that remove humans from meaningful participation in resource allocation.<n>We show that policy myopia is not a symptom of poor attention management but a mechanism producing irreversible human disempowerment.
arXiv Detail & Related papers (2026-03-03T18:54:57Z) - Operational Noncommutativity in Sequential Metacognitive Judgments [0.0]
We show that metacognitive evaluations are modeled as state-transforming operations acting on an internal state space with probabilistic readouts.<n>We also outline a behavioral paradigm involving sequential confidence, error-likelihood, and feeling-of-knowing judgments following a perceptual decision.
arXiv Detail & Related papers (2026-02-15T08:15:48Z) - When Your AI Agent Succumbs to Peer-Pressure: Studying Opinion-Change Dynamics of LLMs [0.0]
We investigate how peer pressure influences the opinions of Large Language Model (LLM) agents across a spectrum of cognitive commitments.<n>Agents follow a sigmoid curve: stable at low pressure, shifting sharply at threshold, and saturating at high.<n>We uncover a fundamental "persuasion asymmetry," where shifting an opinion from affirmative-to-negative requires a different cognitive effort than the reverse.
arXiv Detail & Related papers (2025-10-21T22:02:15Z) - DeceptionBench: A Comprehensive Benchmark for AI Deception Behaviors in Real-world Scenarios [57.327907850766785]
characterization of deception across realistic real-world scenarios remains underexplored.<n>We establish DeceptionBench, the first benchmark that systematically evaluates how deceptive tendencies manifest across different domains.<n>On the intrinsic dimension, we explore whether models exhibit self-interested egoistic tendencies or sycophantic behaviors that prioritize user appeasement.<n>We incorporate sustained multi-turn interaction loops to construct a more realistic simulation of real-world feedback dynamics.
arXiv Detail & Related papers (2025-10-17T10:14:26Z) - Compose and Fuse: Revisiting the Foundational Bottlenecks in Multimodal Reasoning [49.17801010041155]
Multimodal large language models (MLLMs) promise enhanced reasoning by integrating diverse inputs such as text, vision, and audio.<n>Yet cross-modal reasoning remains underexplored, with conflicting reports on whether added modalities help or harm performance.<n>We categorize multimodal reasoning into six interaction patterns, varying how facts are distributed across modalities and logically combined.
arXiv Detail & Related papers (2025-09-28T08:46:11Z) - Disagreements in Reasoning: How a Model's Thinking Process Dictates Persuasion in Multi-Agent Systems [49.69773210844221]
This paper challenges the prevailing hypothesis that persuasive efficacy is primarily a function of model scale.<n>Through a series of multi-agent persuasion experiments, we uncover a fundamental trade-off we term the Persuasion Duality.<n>Our findings reveal that the reasoning process in LRMs exhibits significantly greater resistance to persuasion, maintaining their initial beliefs more robustly.
arXiv Detail & Related papers (2025-09-25T12:03:10Z) - Opacity as Authority: Arbitrariness and the Preclusion of Contestation [0.0]
Article redefines arbitrariness not as a normative flaw or a symptom of domination, but as a functional mechanism structuring human systems and interactions.<n>Building on Ferdinand de Saussure's concept of l'arbitraire du signe, the analysis extends this principle beyond language to demonstrate its cross-domain applicability.
arXiv Detail & Related papers (2025-07-25T19:10:35Z) - A Comment On "The Illusion of Thinking": Reframing the Reasoning Cliff as an Agentic Gap [0.39073867995073247]
We argue that the observed failure is not evidence of a fundamental cognitive boundary, but rather a predictable outcome of system-level constraints.<n>A model, initially declaring a puzzle impossible when confined to text-only generation, now employs agentic tools to not only solve it but also master variations of complexity far beyond the reasoning cliff it previously failed to surmount.
arXiv Detail & Related papers (2025-06-23T17:14:21Z) - Generalization or Hallucination? Understanding Out-of-Context Reasoning in Transformers [76.42159902257677]
We argue that both behaviors stem from a single mechanism known as out-of-context reasoning (OCR)<n>OCR drives both generalization and hallucination, depending on whether the associated concepts are causally related.<n>Our work provides a theoretical foundation for understanding the OCR phenomenon, offering a new lens for analyzing and mitigating undesirable behaviors from knowledge injection.
arXiv Detail & Related papers (2025-06-12T16:50:45Z) - Reciprocity as the Foundational Substrate of Society: How Reciprocal Dynamics Scale into Social Systems [0.0]
We argue that reciprocity is not merely a local or primitive exchange, but a scalable substrate from which large-scale social structures can emerge.<n>We propose a three-stage framework to model this emergence: reciprocal dynamics at the individual level, norm stabilization through shared expectations, and the construction of durable institutional patterns.
arXiv Detail & Related papers (2025-05-13T07:50:01Z) - Emergent Abilities in Large Language Models: A Survey [9.50669909278749]
Large Language Models (LLMs) are leading a new technological revolution as one of the most promising research streams toward artificial general intelligence.<n>The scaling of these models has been linked to various so-called emergent abilities that were previously unobserved.<n>These abilities range from advanced reasoning and in-context learning to coding and problem-solving.<n>Despite their transformative potential, emergent abilities remain poorly understood, leading to misconceptions about their definition, nature, predictability, and implications.
arXiv Detail & Related papers (2025-02-28T01:20:01Z) - Position: Cracking the Code of Cascading Disparity Towards Marginalized Communities [14.497314894554828]
We discuss that disparities towards marginalized communities are not isolated concerns but rather interconnected elements of a cascading disparity phenomenon.
We emphasize the unique threat of cascading impacts in foundation models, where interconnected disparities can trigger long-lasting negative consequences.
We conclude with a set of calls to action to mitigate disparity at its source.
arXiv Detail & Related papers (2024-06-03T19:52:41Z) - Decoding Susceptibility: Modeling Misbelief to Misinformation Through a Computational Approach [61.04606493712002]
Susceptibility to misinformation describes the degree of belief in unverifiable claims that is not observable.
Existing susceptibility studies heavily rely on self-reported beliefs.
We propose a computational approach to model users' latent susceptibility levels.
arXiv Detail & Related papers (2023-11-16T07:22:56Z) - Answering Causal Queries at Layer 3 with DiscoSCMs-Embracing
Heterogeneity [0.0]
This paper advocates for the Distribution-consistency Structural Causal Models (DiscoSCM) framework as a pioneering approach to counterfactual inference.
arXiv Detail & Related papers (2023-09-17T17:01:05Z) - Towards CausalGPT: A Multi-Agent Approach for Faithful Knowledge Reasoning via Promoting Causal Consistency in LLMs [55.66353783572259]
Causal-Consistency Chain-of-Thought harnesses multi-agent collaboration to bolster the faithfulness and causality of foundation models.<n>Our framework demonstrates significant superiority over state-of-the-art methods through extensive and comprehensive evaluations.
arXiv Detail & Related papers (2023-08-23T04:59:21Z) - Towards Trustworthy Explanation: On Causal Rationalization [9.48539398357156]
We propose a new model of rationalization based on two causal desiderata, non-spuriousness and efficiency.
The superior performance of the proposed causal rationalization is demonstrated on real-world review and medical datasets.
arXiv Detail & Related papers (2023-06-25T03:34:06Z) - Causal Triplet: An Open Challenge for Intervention-centric Causal
Representation Learning [98.78136504619539]
Causal Triplet is a causal representation learning benchmark featuring visually more complex scenes.
We show that models built with the knowledge of disentangled or object-centric representations significantly outperform their distributed counterparts.
arXiv Detail & Related papers (2023-01-12T17:43:38Z) - Social Diversity Reduces the Complexity and Cost of Fostering Fairness [63.70639083665108]
We investigate the effects of interference mechanisms which assume incomplete information and flexible standards of fairness.
We quantify the role of diversity and show how it reduces the need for information gathering.
Our results indicate that diversity changes and opens up novel mechanisms available to institutions wishing to promote fairness.
arXiv Detail & Related papers (2022-11-18T21:58:35Z) - Causal Fairness Analysis [68.12191782657437]
We introduce a framework for understanding, modeling, and possibly solving issues of fairness in decision-making settings.
The main insight of our approach will be to link the quantification of the disparities present on the observed data with the underlying, and often unobserved, collection of causal mechanisms.
Our effort culminates in the Fairness Map, which is the first systematic attempt to organize and explain the relationship between different criteria found in the literature.
arXiv Detail & Related papers (2022-07-23T01:06:34Z) - Deconfounding Scores: Feature Representations for Causal Effect
Estimation with Weak Overlap [140.98628848491146]
We introduce deconfounding scores, which induce better overlap without biasing the target of estimation.
We show that deconfounding scores satisfy a zero-covariance condition that is identifiable in observed data.
In particular, we show that this technique could be an attractive alternative to standard regularizations.
arXiv Detail & Related papers (2021-04-12T18:50:11Z) - Cooperation and Reputation Dynamics with Reinforcement Learning [6.219565750197311]
We show how reputations can be used as a way to establish trust and cooperation.
We propose two mechanisms to alleviate convergence to undesirable equilibria.
We show how our results relate to the literature in Evolutionary Game Theory.
arXiv Detail & Related papers (2021-02-15T12:48:56Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.