Exploiting Meta-Cognitive Features for a Machine-Learning-Based One-Shot
Group-Decision Aggregation
- URL: http://arxiv.org/abs/2201.08247v1
- Date: Thu, 20 Jan 2022 15:56:18 GMT
- Title: Exploiting Meta-Cognitive Features for a Machine-Learning-Based One-Shot
Group-Decision Aggregation
- Authors: Hilla Shinitzky, Yuval Shahar, Dan Avraham, Yizhak Vaisman, Yakir
Tsizer and Yaniv Leedon
- Abstract summary: Methods that rely on meta-cognitive information, such as confidence-based methods, had shown an improvement in various tasks.
Our aim is to exploit meta-cognitive information and to learn from it, for the purpose of enhancing the ability of the group to produce a correct answer.
- Score: 0.7340017786387767
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: The outcome of a collective decision-making process, such as crowdsourcing,
often relies on the procedure through which the perspectives of its individual
members are aggregated. Popular aggregation methods, such as the majority rule,
often fail to produce the optimal result, especially in high-complexity tasks.
Methods that rely on meta-cognitive information, such as confidence-based
methods and the Surprisingly Popular Option, had shown an improvement in
various tasks. However, there is still a significant number of cases with no
optimal solution. Our aim is to exploit meta-cognitive information and to learn
from it, for the purpose of enhancing the ability of the group to produce a
correct answer. Specifically, we propose two different feature-representation
approaches: (1) Response-Centered feature Representation (RCR), which focuses
on the characteristics of the individual response instances, and (2)
Answer-Centered feature Representation (ACR), which focuses on the
characteristics of each of the potential answers. Using these two
feature-representation approaches, we train Machine-Learning (ML) models, for
the purpose of predicting the correctness of a response and of an answer. The
trained models are used as the basis of an ML-based aggregation methodology
that, contrary to other ML-based techniques, has the advantage of being a
"one-shot" technique, independent from the crowd-specific composition and
personal record, and adaptive to various types of situations. To evaluate our
methodology, we collected 2490 responses for different tasks, which we used for
feature engineering and for the training of ML models. We tested our
feature-representation approaches through the performance of our proposed
ML-based aggregation methods. The results show an increase of 20% to 35% in the
success rate, compared to the use of standard rule-based aggregation methods.
Related papers
- Interpetable Target-Feature Aggregation for Multi-Task Learning based on Bias-Variance Analysis [53.38518232934096]
Multi-task learning (MTL) is a powerful machine learning paradigm designed to leverage shared knowledge across tasks to improve generalization and performance.
We propose an MTL approach at the intersection between task clustering and feature transformation based on a two-phase iterative aggregation of targets and features.
In both phases, a key aspect is to preserve the interpretability of the reduced targets and features through the aggregation with the mean, which is motivated by applications to Earth science.
arXiv Detail & Related papers (2024-06-12T08:30:16Z) - Self-Supervised Representation Learning with Meta Comprehensive
Regularization [11.387994024747842]
We introduce a module called CompMod with Meta Comprehensive Regularization (MCR), embedded into existing self-supervised frameworks.
We update our proposed model through a bi-level optimization mechanism, enabling it to capture comprehensive features.
We provide theoretical support for our proposed method from information theory and causal counterfactual perspective.
arXiv Detail & Related papers (2024-03-03T15:53:48Z) - A data-science pipeline to enable the Interpretability of Many-Objective
Feature Selection [0.1474723404975345]
Many-Objective Feature Selection (MOFS) approaches use four or more objectives to determine the relevance of a subset of features in a supervised learning task.
This paper proposes an original methodology to support data scientists in the interpretation and comparison of the MOFS outcome by combining post-processing and visualisation of the set of solutions.
arXiv Detail & Related papers (2023-11-30T17:44:22Z) - USER: Unified Semantic Enhancement with Momentum Contrast for Image-Text
Retrieval [115.28586222748478]
Image-Text Retrieval (ITR) aims at searching for the target instances that are semantically relevant to the given query from the other modality.
Existing approaches typically suffer from two major limitations.
arXiv Detail & Related papers (2023-01-17T12:42:58Z) - An Additive Instance-Wise Approach to Multi-class Model Interpretation [53.87578024052922]
Interpretable machine learning offers insights into what factors drive a certain prediction of a black-box system.
Existing methods mainly focus on selecting explanatory input features, which follow either locally additive or instance-wise approaches.
This work exploits the strengths of both methods and proposes a global framework for learning local explanations simultaneously for multiple target classes.
arXiv Detail & Related papers (2022-07-07T06:50:27Z) - Meta-Learning Approaches for a One-Shot Collective-Decision Aggregation:
Correctly Choosing how to Choose Correctly [0.7874708385247353]
We present two one-shot machine-learning-based aggregation approaches.
The first predicts, given multiple features about the collective's choices, which aggregation method will be best for a given case.
The second directly predicts which decision is optimal, given, among other things, the selection made by each method.
arXiv Detail & Related papers (2022-04-03T15:06:59Z) - CAD: Co-Adapting Discriminative Features for Improved Few-Shot
Classification [11.894289991529496]
Few-shot classification is a challenging problem that aims to learn a model that can adapt to unseen classes given a few labeled samples.
Recent approaches pre-train a feature extractor, and then fine-tune for episodic meta-learning.
We propose a strategy to cross-attend and re-weight discriminative features for few-shot classification.
arXiv Detail & Related papers (2022-03-25T06:14:51Z) - Variational Empowerment as Representation Learning for Goal-Based
Reinforcement Learning [114.07623388322048]
We discuss how the standard goal-conditioned RL (GCRL) is encapsulated by the objective variational empowerment.
Our work lays a novel foundation from which to evaluate, analyze, and develop representation learning techniques in goal-based RL.
arXiv Detail & Related papers (2021-06-02T18:12:26Z) - Leveraging Expert Consistency to Improve Algorithmic Decision Support [62.61153549123407]
We explore the use of historical expert decisions as a rich source of information that can be combined with observed outcomes to narrow the construct gap.
We propose an influence function-based methodology to estimate expert consistency indirectly when each case in the data is assessed by a single expert.
Our empirical evaluation, using simulations in a clinical setting and real-world data from the child welfare domain, indicates that the proposed approach successfully narrows the construct gap.
arXiv Detail & Related papers (2021-01-24T05:40:29Z) - Task-Feature Collaborative Learning with Application to Personalized
Attribute Prediction [166.87111665908333]
We propose a novel multi-task learning method called Task-Feature Collaborative Learning (TFCL)
Specifically, we first propose a base model with a heterogeneous block-diagonal structure regularizer to leverage the collaborative grouping of features and tasks.
As a practical extension, we extend the base model by allowing overlapping features and differentiating the hard tasks.
arXiv Detail & Related papers (2020-04-29T02:32:04Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.