Evaluating Pluralism in LLMs through Latent Perspectives

• URL: http://arxiv.org/abs/2606.13254v1
• Date: Thu, 11 Jun 2026 12:11:04 GMT
• Title: Evaluating Pluralism in LLMs through Latent Perspectives
• Abstract summary: We introduce and implement a domain-agnostic multi-layered framework for unsupervised extraction of perspectives suitable for identifying the pluralistic gap in LLM-generated text.<n>Our results show that while some models and prompting techniques come close to covering a broad spectrum of perspectives, rarer perspectives remain disproportionately underrepresented.
• Score: 3.4155322317700576
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: The growing need to represent diverse perspectives has increased interest in pluralistic LLM generation. Although difficult to operationalize, identifying perspectives expressed in text would provide clear guidance on pluralistic alignment and more clearly articulate the pluralistic gap in LLM generation. While models have been shown to reduce the diversity of training data and generate homogeneously, this has been demonstrated primarily on multiple-choice questionnaires or using high-level characteristics of free-form text. In this paper, we introduce and implement a domain-agnostic multi-layered framework for unsupervised extraction of perspectives suitable for identifying the pluralistic gap in LLM-generated text. We evaluate our framework on book reviews, a highly opinionated dataset representing diverse perspectives, and compare various prompts and models. Our results show that while some models and prompting techniques come close to covering a broad spectrum of perspectives, rarer perspectives remain disproportionately underrepresented, resulting in distributions that diverge from human text.

Related papers

• Explicit Representation Alignment for Multimodal Sentiment Analysis [17.25685888421759]
  Multimodal affective analysis aims to understand human sentiment and emotion by jointly modeling heterogeneous modalities such as text and images.<n>We identify representation misalignment between independently pretrained modality encoders as a key bottleneck for effective multimodal learning.<n>We propose a unified multimodal affective analysis framework that leverages vision-language models (VLMs) to convert visual content into structured textual descriptions.
  arXiv  Detail & Related papers  (2026-06-08T07:43:52Z)
• Bridging Lexical Ambiguity and Vision: A Mini Review on Visual Word Sense Disambiguation [0.0]
  Visual Word Sense Disambiguation helps tackle lexical ambiguity in vision-language tasks.<n>VWSD uses visual cues to find the right meaning of ambiguous words with minimal text input.<n>Studies from 2016 to 2025 are examined to show the growth of VWSD through feature-based, graph-based, and contrastive embedding techniques.
  arXiv  Detail & Related papers  (2026-02-01T12:36:01Z)
• Where Does Vision Meet Language? Understanding and Refining Visual Fusion in MLLMs via Contrastive Attention [7.511262066889113]
  Multimodal Large Language Models (MLLMs) have achieved remarkable progress in vision-language understanding.<n>We perform a systematic layer-wise masking analysis across multiple architectures, revealing how visual-text fusion evolves within MLLMs.<n>We introduce a training-free contrastive attention framework that models the transformation between early fusion and final layers to highlight meaningful attention shifts.
  arXiv  Detail & Related papers  (2026-01-13T02:26:21Z)
• Rethinking the Text-Vision Reasoning Imbalance in MLLMs through the Lens of Training Recipes [54.374410871041164]
  Multimodal large language models (MLLMs) have demonstrated strong capabilities on vision-and-language tasks.<n>Recent findings reveal an imbalance in their reasoning capabilities across visual and textual modalities.<n>We refer to this phenomenon as the textitmodality gap, defined as the performance disparity between text-centric and vision-centric inputs.
  arXiv  Detail & Related papers  (2025-10-26T21:06:13Z)
• Perspectives in Play: A Multi-Perspective Approach for More Inclusive NLP Systems [3.011820285006942]
  This study proposes a new multi-perspective approach using soft labels to encourage the development of perspective aware models.<n>We conduct an analysis across diverse subjective text classification tasks, including hate speech, irony, abusive language, and stance detection.<n>Results show that the multi-perspective approach better approximates human label distributions, as measured by Jensen-Shannon Divergence (JSD)<n>Our approach exhibits lower confidence in tasks like irony and stance detection, likely due to the inherent subjectivity present in the texts.
  arXiv  Detail & Related papers  (2025-06-25T07:53:36Z)
• Multi-Novelty: Improve the Diversity and Novelty of Contents Generated by Large Language Models via inference-time Multi-Views Brainstorming [3.591342811819669]
  Large Language Models (LLMs) demonstrate remarkable proficiency in generating accurate and fluent text.<n>They often struggle with diversity and novelty, leading to repetitive or overly deterministic responses.<n>We introduce inference-time multi-view brainstorming method, a novel approach that enriches input prompts with diverse perspectives.
  arXiv  Detail & Related papers  (2025-02-18T10:04:20Z)
• ViLBias: Detecting and Reasoning about Bias in Multimodal Content [6.710013984078675]
  ViLBias is a framework for detecting and reasoning about bias in multimodal news.<n> dataset comprises 40,945 text--image pairs from diverse outlets.<n>Results show that images alongside text improves detection accuracy by 3--5%.
  arXiv  Detail & Related papers  (2024-12-22T15:05:30Z)
• Unified Generative and Discriminative Training for Multi-modal Large Language Models [88.84491005030316]
  Generative training has enabled Vision-Language Models (VLMs) to tackle various complex tasks. Discriminative training, exemplified by models like CLIP, excels in zero-shot image-text classification and retrieval. This paper proposes a unified approach that integrates the strengths of both paradigms.
  arXiv  Detail & Related papers  (2024-11-01T01:51:31Z)
• Multi-modal Instruction Tuned LLMs with Fine-grained Visual Perception [63.03288425612792]
  We propose bfAnyRef, a general MLLM model that can generate pixel-wise object perceptions and natural language descriptions from multi-modality references. Our model achieves state-of-the-art results across multiple benchmarks, including diverse modality referring segmentation and region-level referring expression generation.
  arXiv  Detail & Related papers  (2024-03-05T13:45:46Z)
• Exploring Precision and Recall to assess the quality and diversity of LLMs [82.21278402856079]
  We introduce a novel evaluation framework for Large Language Models (LLMs) such as textscLlama-2 and textscMistral. This approach allows for a nuanced assessment of the quality and diversity of generated text without the need for aligned corpora.
  arXiv  Detail & Related papers  (2024-02-16T13:53:26Z)
• How Far Can We Extract Diverse Perspectives from Large Language Models? [16.16678226707335]
  We show that large language models (LLMs) can generate diverse perspectives on subjective topics. We propose a criteria-based prompting technique to ground diverse opinions. Our methods, applied to various tasks, show that LLMs can indeed produce diverse opinions according to the degree of task subjectivity.
  arXiv  Detail & Related papers  (2023-11-16T11:23:38Z)
• Large Language Models can Contrastively Refine their Generation for Better Sentence Representation Learning [57.74233319453229]
  Large language models (LLMs) have emerged as a groundbreaking technology and their unparalleled text generation capabilities have sparked interest in their application to the fundamental sentence representation learning task. We propose MultiCSR, a multi-level contrastive sentence representation learning framework that decomposes the process of prompting LLMs to generate a corpus. Our experiments reveal that MultiCSR enables a less advanced LLM to surpass the performance of ChatGPT, while applying it to ChatGPT achieves better state-of-the-art results.
  arXiv  Detail & Related papers  (2023-10-17T03:21:43Z)
• OCRBench: On the Hidden Mystery of OCR in Large Multimodal Models [122.27878464009181]
  We conducted a comprehensive evaluation of Large Multimodal Models, such as GPT4V and Gemini, in various text-related visual tasks. OCRBench contains 29 datasets, making it the most comprehensive OCR evaluation benchmark available.
  arXiv  Detail & Related papers  (2023-05-13T11:28:37Z)
• Deep Partial Multi-View Learning [94.39367390062831]
  We propose a novel framework termed Cross Partial Multi-View Networks (CPM-Nets) We fifirst provide a formal defifinition of completeness and versatility for multi-view representation. We then theoretically prove the versatility of the learned latent representations.
  arXiv  Detail & Related papers  (2020-11-12T02:29:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.