Social Comparison without Explicit Inference of Others' Reward Values: A Constructive Approach Using a Probabilistic Generative Model

• URL: http://arxiv.org/abs/2512.18687v2
• Date: Tue, 23 Dec 2025 06:51:30 GMT
• Title: Social Comparison without Explicit Inference of Others' Reward Values: A Constructive Approach Using a Probabilistic Generative Model
• Authors: Yosuke Taniuchi, Chie Hieida, Atsushi Noritake, Kazushi Ikeda, Masaki Isoda,
• Abstract summary: Social comparison relies on objective reward differences rather than inferences about subjective states.<n>We train models on a dataset containing a pair of monkeys, their rewards, and stimuli.<n>We evaluate the models' ability to classify subjective values across pre-defined experimental conditions.
• Score: 1.9732490977700972
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Social comparison$\unicode{x2014}$the process of evaluating one's rewards relative to others$\unicode{x2014}$plays a fundamental role in primate social cognition. However, it remains unknown from a computational perspective how information about others' rewards affects the evaluation of one's own reward. With a constructive approach, this study examines whether monkeys merely recognize objective reward differences or, instead, infer others' subjective reward valuations. We developed three computational models with varying degrees of social information processing: an Internal Prediction Model (IPM), which infers the partner's subjective values; a No Comparison Model (NCM), which disregards partner information; and an External Comparison Model (ECM), which directly incorporates the partner's objective rewards. To test model performance, we used a multi-layered, multimodal latent Dirichlet allocation. We trained the models on a dataset containing the behavior of a pair of monkeys, their rewards, and the conditioned stimuli. Then, we evaluated the models' ability to classify subjective values across pre-defined experimental conditions. The ECM achieved the highest classification score in the Rand Index (0.88 vs. 0.79 for the IPM) under our settings, suggesting that social comparison relies on objective reward differences rather than inferences about subjective states.

Related papers

• Modeling Art Evaluations from Comparative Judgments: A Deep Learning Approach to Predicting Aesthetic Preferences [1.839031891198526]
  Law of Comparative Judgment posits that relative choices exhibit less cognitive burden and greater cognitive consistency than direct scoring.<n>We develop a deep neural network regression model and a dual-branch pairwise comparison model.<n>Human subject experiments reveal that comparative judgments require $60%$ less annotation time per item.
  arXiv  Detail & Related papers  (2026-01-30T23:13:06Z)
• Probing Preference Representations: A Multi-Dimensional Evaluation and Analysis Method for Reward Models [63.00458229517523]
  This work addresses the evaluation challenge of reward models by probing preference representations.<n>We construct a Multi-dimensional Reward Model Benchmark (MRMBench), a collection of six probing tasks for different preference dimensions.<n>We introduce an analysis method, inference-time probing, which identifies the dimensions used during the reward prediction and enhances its interpretability.
  arXiv  Detail & Related papers  (2025-11-16T05:29:29Z)
• A Ratio-Based Shapley Value for Collaborative Machine Learning - Extended Version [1.7778609937758325]
  Collaborative machine learning enables multiple data owners to jointly train models for improved predictive performance.<n> Ensuring incentive compatibility and fair contribution-based rewards remains a critical challenge.<n>We introduce a ratio-based Shapley value that replaces the standard additive formulation with a relative contribution measure.
  arXiv  Detail & Related papers  (2025-10-15T08:08:18Z)
• From Scores to Preferences: Redefining MOS Benchmarking for Speech Quality Reward Modeling [66.22134521383909]
  We introduce a unified benchmark that reformulates diverse MOS datasets into a preference-comparison setting.<n>Building on MOS-RMBench, we systematically construct and evaluate three paradigms for reward modeling.<n>Our experiments reveal three key findings: (1) scalar models achieve the strongest overall performance, consistently exceeding 74% accuracy; (2) most models perform considerably worse on synthetic speech than on human speech; and (3) all models struggle on pairs with very small MOS differences.<n> Experimental results show that the MOS-aware GRM significantly improves fine-grained quality discrimination and narrows the gap with scalar models on the most challenging cases.
  arXiv  Detail & Related papers  (2025-10-01T10:27:51Z)
• CPO: Addressing Reward Ambiguity in Role-playing Dialogue via Comparative Policy Optimization [53.79487826635141]
  Reinforcement Learning Fine-Tuning (RLFT) has achieved notable success in tasks with objectively verifiable answers.<n>But it struggles with open-ended subjective tasks like role-playing dialogue.<n>Traditional reward modeling approaches, which rely on independent sample-wise scoring, face dual challenges: subjective evaluation criteria and unstable reward signals.<n>Motivated by the insight that human evaluation inherently combines explicit criteria with implicit comparative judgments, we propose Comparative Policy Optimization.
  arXiv  Detail & Related papers  (2025-08-12T16:49:18Z)
• EvolvR: Self-Evolving Pairwise Reasoning for Story Evaluation to Enhance Generation [17.37840331449749]
  We propose a self-Evolving Pairwise Reasoning (EvolvR) framework for story evaluation.<n>The framework first self-synthesizes score-aligned Chain-of-Thought (CoT) data via a multi-persona strategy.<n>The evaluator trained on the refined data is deployed as a reward model to guide the story generation task.
  arXiv  Detail & Related papers  (2025-08-08T06:10:47Z)
• Rethinking Reward Model Evaluation Through the Lens of Reward Overoptimization [15.729285736811383]
  Reward models play a crucial role in reinforcement learning from human feedback.<n>Existing benchmarks for reward models show a weak correlation with the performance of optimized policies.
  arXiv  Detail & Related papers  (2025-05-19T06:43:08Z)
• Where is this coming from? Making groundedness count in the evaluation of Document VQA models [12.951716701565019]
  We argue that common evaluation metrics do not account for the semantic and multimodal groundedness of a model's outputs.<n>We propose a new evaluation methodology that accounts for the groundedness of predictions.<n>Our proposed methodology is parameterized in such a way that users can configure the score according to their preferences.
  arXiv  Detail & Related papers  (2025-03-24T20:14:46Z)
• SureMap: Simultaneous Mean Estimation for Single-Task and Multi-Task Disaggregated Evaluation [75.56845750400116]
  Disaggregated evaluation -- estimation of performance of a machine learning model on different subpopulations -- is a core task when assessing performance and group-fairness of AI systems. We develop SureMap that has high estimation accuracy for both multi-task and single-task disaggregated evaluations of blackbox models. Our method combines maximum a posteriori (MAP) estimation using a well-chosen prior together with cross-validation-free tuning via Stein's unbiased risk estimate (SURE)
  arXiv  Detail & Related papers  (2024-11-14T17:53:35Z)
• Evaluating Robustness of Reward Models for Mathematical Reasoning [14.97819343313859]
  We introduce a new design for reliable evaluation of reward models, and to validate this, we construct RewardMATH. We demonstrate that the scores on RewardMATH strongly correlate with the results of optimized policy and effectively estimate reward overoptimization.
  arXiv  Detail & Related papers  (2024-10-02T16:39:58Z)
• RewardBench: Evaluating Reward Models for Language Modeling [100.28366840977966]
  We present RewardBench, a benchmark dataset and code-base for evaluation of reward models. The dataset is a collection of prompt-chosen-rejected trios spanning chat, reasoning, and safety. On the RewardBench leaderboard, we evaluate reward models trained with a variety of methods.
  arXiv  Detail & Related papers  (2024-03-20T17:49:54Z)
• Differentially Private Reward Estimation with Preference Feedback [15.943664678210146]
  Learning from preference-based feedback has recently gained considerable traction as a promising approach to align generative models with human interests. An adversarial attack in any step of the above pipeline might reveal private and sensitive information of human labelers. We focus on the problem of reward estimation from preference-based feedback while protecting privacy of each individual labelers.
  arXiv  Detail & Related papers  (2023-10-30T16:58:30Z)
• Evaluating the Fairness of Discriminative Foundation Models in Computer Vision [51.176061115977774]
  We propose a novel taxonomy for bias evaluation of discriminative foundation models, such as Contrastive Language-Pretraining (CLIP) We then systematically evaluate existing methods for mitigating bias in these models with respect to our taxonomy. Specifically, we evaluate OpenAI's CLIP and OpenCLIP models for key applications, such as zero-shot classification, image retrieval and image captioning.
  arXiv  Detail & Related papers  (2023-10-18T10:32:39Z)
• GREAT Score: Global Robustness Evaluation of Adversarial Perturbation using Generative Models [60.48306899271866]
  We present a new framework, called GREAT Score, for global robustness evaluation of adversarial perturbation using generative models. We show high correlation and significantly reduced cost of GREAT Score when compared to the attack-based model ranking on RobustBench. GREAT Score can be used for remote auditing of privacy-sensitive black-box models.
  arXiv  Detail & Related papers  (2023-04-19T14:58:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.