AI Alignment From Social Choice Perspectives

• URL: http://arxiv.org/abs/2606.21550v1
• Date: Fri, 19 Jun 2026 15:47:01 GMT
• Title: AI Alignment From Social Choice Perspectives
• Abstract summary: We show how social choice theory helps identify failure modes in the feedback aggregation layer.<n>We show how the social choice perspective helps identify a broader design space for handling disagreement in explicit and principled ways.
• Score: 37.88586337953354
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Alignment from human feedback uses human judgments about model outputs to steer the behavior of language models after pretraining. When those judgments reflect conflicting views of desirable behavior, the learned objective becomes an aggregate determination of what the model should prefer. We survey recent work that has studied this aggregation problem through the lens of social choice theory. We illustrate how the social choice perspective helps identify failure modes in the feedback aggregation layer and reveals a broader design space for handling disagreement in explicit and principled ways.

Related papers

• Beyond Sycophancy: Structured Resistance and Compliance in LLM Moral Reasoning [0.42970700836450476]
  We show that models' judgment revision is structured along three dimensions that parallel classic phenomena in human social psychology.<n>We recast sycophancy as one expression of a broader judgment-updating process shaped by social influence.
  arXiv  Detail & Related papers  (2026-07-23T17:40:07Z)
• Modeling Human Perspectives with Socio-Demographic Representations [0.6138671548064355]
  We propose Socio-Contrastive Learning, a method that jointly models annotator perspectives while learning socio-demographic representations.<n>Our method provides an effective approach for the fusion of socio-demographic features and textual representations to predict annotator perspectives.
  arXiv  Detail & Related papers  (2026-04-20T10:37:26Z)
• Measuring Social Bias in Vision-Language Models with Face-Only Counterfactuals from Real Photos [79.03150233804458]
  Real-world images entangle race and gender with correlated factors such as background and clothing, obscuring attribution.<n>We propose a textbfface-only counterfactual evaluation paradigm<n>We generate counterfactual variants by editing only facial attributes related to race and gender, keeping all other visual factors fixed.
  arXiv  Detail & Related papers  (2026-01-11T14:35:06Z)
• EMBRACE: Shaping Inclusive Opinion Representation by Aligning Implicit Conversations with Social Norms [0.14323566945483493]
  This study aims to evaluate how opinions are represented in NLP or computational models by introducing an alignment evaluation framework.<n>Our approach models the stance of responses as a proxy for the underlying opinion, enabling a considerate and reflective representation of diverse social viewpoints.
  arXiv  Detail & Related papers  (2025-07-27T13:21:07Z)
• Preference Learning for AI Alignment: a Causal Perspective [55.2480439325792]
  We frame this problem in a causal paradigm, providing the rich toolbox of causality to identify persistent challenges.<n>Inheriting from the literature of causal inference, we identify key assumptions necessary for reliable generalisation.<n>We illustrate failure modes of naive reward models and demonstrate how causally-inspired approaches can improve model robustness.
  arXiv  Detail & Related papers  (2025-06-06T10:45:42Z)
• Diverging Preferences: When do Annotators Disagree and do Models Know? [92.24651142187989]
  We develop a taxonomy of disagreement sources spanning 10 categories across four high-level classes. We find that the majority of disagreements are in opposition with standard reward modeling approaches. We develop methods for identifying diverging preferences to mitigate their influence on evaluation and training.
  arXiv  Detail & Related papers  (2024-10-18T17:32:22Z)
• Inverse Decision Modeling: Learning Interpretable Representations of Behavior [72.80902932543474]
  We develop an expressive, unifying perspective on inverse decision modeling. We use this to formalize the inverse problem (as a descriptive model) We illustrate how this structure enables learning (interpretable) representations of (bounded) rationality.
  arXiv  Detail & Related papers  (2023-10-28T05:05:01Z)
• Bridging the Gap: A Survey on Integrating (Human) Feedback for Natural Language Generation [68.9440575276396]
  This survey aims to provide an overview of the recent research that has leveraged human feedback to improve natural language generation. First, we introduce an encompassing formalization of feedback, and identify and organize existing research into a taxonomy following this formalization. Second, we discuss how feedback can be described by its format and objective, and cover the two approaches proposed to use feedback (either for training or decoding): directly using the feedback or training feedback models. Third, we provide an overview of the nascent field of AI feedback, which exploits large language models to make judgments based on a set of principles and minimize the need for
  arXiv  Detail & Related papers  (2023-05-01T17:36:06Z)
• Towards Explaining Subjective Ground of Individuals on Social Media [28.491401997248527]
  This research proposes a neural model that learns subjective grounds of individuals and accounts for their judgments on situations of others posted on social media. Using simple attention modules as well as taking one's previous activities into consideration, we empirically show that our model provides human-readable explanations of an individual's subjective preference in judging social situations.
  arXiv  Detail & Related papers  (2022-11-18T00:29:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.