Value Preferences Estimation and Disambiguation in Hybrid Participatory Systems

• URL: http://arxiv.org/abs/2402.16751v2
• Date: Tue, 29 Oct 2024 09:25:30 GMT
• Title: Value Preferences Estimation and Disambiguation in Hybrid Participatory Systems
• Authors: Enrico Liscio, Luciano C. Siebert, Catholijn M. Jonker, Pradeep K. Murukannaiah,
• Abstract summary: We envision a hybrid participatory system where participants make choices and provide motivations for those choices. We focus on situations where a conflict is detected between participants' choices and motivations. We propose methods for estimating value preferences while addressing detected inconsistencies by interacting with the participants.
• Score: 3.7846812749505134
• License:
• Abstract: Understanding citizens' values in participatory systems is crucial for citizen-centric policy-making. We envision a hybrid participatory system where participants make choices and provide motivations for those choices, and AI agents estimate their value preferences by interacting with them. We focus on situations where a conflict is detected between participants' choices and motivations, and propose methods for estimating value preferences while addressing detected inconsistencies by interacting with the participants. We operationalize the philosophical stance that ``valuing is deliberatively consequential." That is, if a participant's choice is based on a deliberation of value preferences, the value preferences can be observed in the motivation the participant provides for the choice. Thus, we propose and compare value preferences estimation methods that prioritize the values estimated from motivations over the values estimated from choices alone. Then, we introduce a disambiguation strategy that combines Natural Language Processing and Active Learning to address the detected inconsistencies between choices and motivations. We evaluate the proposed methods on a dataset of a large-scale survey on energy transition. The results show that explicitly addressing inconsistencies between choices and motivations improves the estimation of an individual's value preferences. The disambiguation strategy does not show substantial improvements when compared to similar baselines -- however, we discuss how the novelty of the approach can open new research avenues and propose improvements to address the current limitations.

Related papers

• Diverging Preferences: When do Annotators Disagree and do Models Know? [92.24651142187989]
  We develop a taxonomy of disagreement sources spanning 10 categories across four high-level classes. We find that the majority of disagreements are in opposition with standard reward modeling approaches. We develop methods for identifying diverging preferences to mitigate their influence on evaluation and training.
  arXiv  Detail & Related papers  (2024-10-18T17:32:22Z)
• (De)Noise: Moderating the Inconsistency Between Human Decision-Makers [15.291993233528526]
  We study whether algorithmic decision aids can be used to moderate the degree of inconsistency in human decision-making in the context of real estate appraisal. We find that both (i) asking respondents to review their estimates in a series of algorithmically chosen pairwise comparisons and (ii) providing respondents with traditional machine advice are effective strategies for influencing human responses.
  arXiv  Detail & Related papers  (2024-07-15T20:24:36Z)
• Be Aware of the Neighborhood Effect: Modeling Selection Bias under Interference [50.95521705711802]
  Previous studies have focused on addressing selection bias to achieve unbiased learning of the prediction model. This paper formally formulates the neighborhood effect as an interference problem from the perspective of causal inference. We propose a novel ideal loss that can be used to deal with selection bias in the presence of neighborhood effect.
  arXiv  Detail & Related papers  (2024-04-30T15:20:41Z)
• Evaluating the Fairness of Discriminative Foundation Models in Computer Vision [51.176061115977774]
  We propose a novel taxonomy for bias evaluation of discriminative foundation models, such as Contrastive Language-Pretraining (CLIP) We then systematically evaluate existing methods for mitigating bias in these models with respect to our taxonomy. Specifically, we evaluate OpenAI's CLIP and OpenCLIP models for key applications, such as zero-shot classification, image retrieval and image captioning.
  arXiv  Detail & Related papers  (2023-10-18T10:32:39Z)
• Subjective-objective policy making approach: Coupling of resident-values multiple regression analysis with value-indices, multi-agent-based simulation [0.0]
  This study proposes a new combined subjective-objective policy evaluation approach to choose better policy. The proposed approach establishes a subjective target function based on a multiple regression analysis of the results of a residents questionnaire survey. Using the new approach to compare several policies enables concrete expression of the will of stakeholders with diverse values.
  arXiv  Detail & Related papers  (2023-06-14T02:33:32Z)
• In Search of Insights, Not Magic Bullets: Towards Demystification of the Model Selection Dilemma in Heterogeneous Treatment Effect Estimation [92.51773744318119]
  This paper empirically investigates the strengths and weaknesses of different model selection criteria. We highlight that there is a complex interplay between selection strategies, candidate estimators and the data used for comparing them.
  arXiv  Detail & Related papers  (2023-02-06T16:55:37Z)
• Inclusive Artificial Intelligence [27.09425461169165]
  Methods for assessing and comparing generative AIs incentivize responses that serve a hypothetical representative individual. We propose an alternative evaluation method that instead prioritizes inclusive AIs.
  arXiv  Detail & Related papers  (2022-12-24T02:13:26Z)
• Justification of Recommender Systems Results: A Service-based Approach [4.640835690336653]
  We propose a novel justification approach that uses service models to extract experience data from reviews concerning all the stages of interaction with items. In a user study, we compared our approach with baselines reflecting the state of the art in the justification of recommender systems results. Our models received higher Interface Adequacy and Satisfaction evaluations by users having different levels of Curiosity or low Need for Cognition (NfC) These findings encourage the adoption of service models to justify recommender systems results but suggest the investigation of personalization strategies to suit diverse interaction needs.
  arXiv  Detail & Related papers  (2022-11-07T11:08:19Z)
• Models of human preference for learning reward functions [80.39289349661364]
  We learn the reward function from human-generated preferences between pairs of trajectory segments. We find this assumption to be flawed and propose modeling human preferences as informed by each segment's regret. Our proposed regret preference model better predicts real human preferences and also learns reward functions from these preferences that lead to policies that are better human-aligned.
  arXiv  Detail & Related papers  (2022-06-05T17:58:02Z)
• Demarcating Endogenous and Exogenous Opinion Dynamics: An Experimental Design Approach [27.975266406080152]
  In this paper, we design a suite of unsupervised classification methods based on experimental design approaches. We aim to select the subsets of events which minimize different measures of mean estimation error. Our experiments range from validating prediction performance on unsanitized and sanitized events to checking the effect of selecting optimal subsets of various sizes.
  arXiv  Detail & Related papers  (2021-02-11T11:38:15Z)
• Inverse Active Sensing: Modeling and Understanding Timely Decision-Making [111.07204912245841]
  We develop a framework for the general setting of evidence-based decision-making under endogenous, context-dependent time pressure. We demonstrate how it enables modeling intuitive notions of surprise, suspense, and optimality in decision strategies.
  arXiv  Detail & Related papers  (2020-06-25T02:30:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.