Sixteen models, fewer than two voices: measuring ensemble dispersion where no answer is uniquely correct

• URL: http://arxiv.org/abs/2608.00285v1
• Date: Fri, 31 Jul 2026 20:42:53 GMT
• Title: Sixteen models, fewer than two voices: measuring ensemble dispersion where no answer is uniquely correct
• Abstract summary: Sixteen language models drawn from ten families produced, on average, the semantic diversity of 1.69 distinct formulations of a psychotherapeutic case.<n> Dispersion over their outputs is measured both as diversity and as uncertainty.
• Score: 37.64799781700206
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Sixteen language models drawn from ten families produced, on average, the semantic diversity of 1.69 distinct formulations of a psychotherapeutic case, against a single-model baseline of 1.43 from one model's own runs. Ensembles place more than one reading before a decision-maker on the premise that several models supply several perspectives. Dispersion over their outputs is measured both as diversity and as uncertainty, and both traditions validate it against a correctness criterion that this task does not admit. Measuring diversity is a solved problem: the Vendi Score, the exponential of the von Neumann entropy of a similarity matrix, is an effective number of distinct elements. What a single aggregate does not say is where the diversity comes from. We define a per-model dissent contribution, the complement of a model's mean similarity to the other members of its ensemble: a magnitude from the same matrix, not a decomposition of the spectral index, whose maximum identifies the most divergent voice. Crossing model and case, we test as a preregistered hypothesis whether model identity accounts for a non-zero share of the variance in dissent, and characterise the structure that test detects. The panel formulated fifteen stratified vignettes, yielding 7,082 formulations for analysis. Model identity was a detectable structuring factor of the dissent that remained, but the usual categories recovered it only partly: scale differences pointed in opposite directions across pairs, family grouped models on only five two-member lines, and the most divergent voice changed with panel composition, so that the surfaced outlier describes the ensemble rather than the model. Dissent did not track the interpretive openness for which the case bank was stratified; it was organised by clinical content instead, leaving the dispersion an ensemble produces a property to measure rather than assume.

Related papers

• Inferred Generative-Process Diversity Predicts Correlated Failure Across Language Models [1.4323566945483497]
  We show that generative-process diversity is associated with reduced correlated failure in model pairs that is not attributable to semantic similarity or capability.<n>Inferred generative-process diversity offers a novel and practical approach for investigating diversity of multi-model systems in safety-relevant contexts.
  arXiv  Detail & Related papers  (2026-09-03T06:27:58Z)
• When Is a General Factor Distinguishable? Non-Proportionality, Stable Structure, and the Bifactor Decision [2.1494179586067537]
  Distinguishability is graded, measured by the population distance to the $K$-factor class.<n>Four empirical datasets illustrate the possible outcomes.
  arXiv  Detail & Related papers  (2026-08-11T09:46:15Z)
• CogArena: A Multimethod Evaluation of Cognitive Ability Structure in Large Language Models [3.991518853474585]
  We introduce CogArena, a benchmark for determining when cognitive-task scores warrant dimensional labels.<n>Across 55 open-weight models, nearly all paradigm correlations are positive and a common axis explains about half the variance.<n>The within-grouping advantage is small, scoring-sensitive, and uncertain across model families.
  arXiv  Detail & Related papers  (2026-07-27T18:56:13Z)
• Traits Run Deeper: Trait-Specific Asymmetric Fusion for Personality Assessment [53.50824432699408]
  Traits Run Deeper is a novel personality assessment framework.<n>It consists of three components: the Multimodal Foundation Representation (MFR), the Trait-Specific Modality Fusion (TSMF) and the Distribution-Calibrated Personality Regression (DCPR)
  arXiv  Detail & Related papers  (2026-06-09T06:38:36Z)
• LLMs Show No Signs Of Individuated Metacognition [0.023227405857540805]
  We decompose binary confidence judgements from 20 Large Language Models.<n>We ask whether two models that differ in confidence also differ in performance.<n>We find no evidence for significant verbalised individuated metacognition in any tested domain.
  arXiv  Detail & Related papers  (2026-05-22T23:54:33Z)
• A Mechanistic Study of Tabular Foundation Models [12.26738662548555]
  Tabular foundation models converge in accuracy across a range of classification and regression tasks.<n>This raises questions a leaderboard cannot answer.<n>We characterize all three: (i) whether the models execute the same in-context algorithm, (ii) where row, column, and class-permutation invariances originate, and (iii) how robust they are under perturbations engineered against the inferred mechanism.
  arXiv  Detail & Related papers  (2026-05-20T15:23:16Z)
• Identification of Causal Direction under an Arbitrary Number of Latent Confounders [54.76982125821112]
  In real-world scenarios, observed variables may be affected by multiple latent variables simultaneously.<n>We make use of the joint higher-order cumulant matrix of the observed variables constructed in a specific way.<n>We show that, surprisingly, causal asymmetry between two observed variables can be directly seen from the rank deficiency properties of such higher-order cumulant matrices.
  arXiv  Detail & Related papers  (2025-10-26T15:10:00Z)
• Counting Like Human: Anthropoid Crowd Counting on Modeling the Similarity of Objects [92.80955339180119]
  mainstream crowd counting methods regress density map and integrate it to obtain counting results. Inspired by this, we propose a rational and anthropoid crowd counting framework.
  arXiv  Detail & Related papers  (2022-12-02T07:00:53Z)
• Causal Discovery in Linear Latent Variable Models Subject to Measurement Error [29.78435955758185]
  We focus on causal discovery in the presence of measurement error in linear systems. We demonstrate a surprising connection between this problem and causal discovery in the presence of unobserved parentless causes.
  arXiv  Detail & Related papers  (2022-11-08T03:43:14Z)
• Predicting Out-of-Domain Generalization with Neighborhood Invariance [59.05399533508682]
  We propose a measure of a classifier's output invariance in a local transformation neighborhood. Our measure is simple to calculate, does not depend on the test point's true label, and can be applied even in out-of-domain (OOD) settings. In experiments on benchmarks in image classification, sentiment analysis, and natural language inference, we demonstrate a strong and robust correlation between our measure and actual OOD generalization.
  arXiv  Detail & Related papers  (2022-07-05T14:55:16Z)
• Learning Disentangled Representations with Latent Variation Predictability [102.4163768995288]
  This paper defines the variation predictability of latent disentangled representations. Within an adversarial generation process, we encourage variation predictability by maximizing the mutual information between latent variations and corresponding image pairs. We develop an evaluation metric that does not rely on the ground-truth generative factors to measure the disentanglement of latent representations.
  arXiv  Detail & Related papers  (2020-07-25T08:54:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.