A Truthful Owner-Assisted Scoring Mechanism
- URL: http://arxiv.org/abs/2206.08149v1
- Date: Tue, 14 Jun 2022 14:35:53 GMT
- Title: A Truthful Owner-Assisted Scoring Mechanism
- Authors: Weijie J. Su
- Abstract summary: We show that Bob can obtain an estimator with the optimal squared error in certain regimes based on truthful information elicitation.
The estimated grades are substantially more accurate than the raw grades when the number of items is large and the raw grades are very noisy.
- Score: 28.364491520646084
- License: http://creativecommons.org/licenses/by-sa/4.0/
- Abstract: Alice (owner) has knowledge of the underlying quality of her items measured
in grades. Given the noisy grades provided by an independent party, can Bob
(appraiser) obtain accurate estimates of the ground-truth grades of the items
by asking Alice a question about the grades? We address this when the payoff to
Alice is additive convex utility over all her items. We establish that if Alice
has to truthfully answer the question so that her payoff is maximized, the
question must be formulated as pairwise comparisons between her items. Next, we
prove that if Alice is required to provide a ranking of her items, which is the
most fine-grained question via pairwise comparisons, she would be truthful. By
incorporating the ground-truth ranking, we show that Bob can obtain an
estimator with the optimal squared error in certain regimes based on any
possible way of truthful information elicitation. Moreover, the estimated
grades are substantially more accurate than the raw grades when the number of
items is large and the raw grades are very noisy. Finally, we conclude the
paper with several extensions and some refinements for practical
considerations.
Related papers
- Additive-Effect Assisted Learning [17.408937094829007]
We develop a two-stage assisted learning architecture for an agent, Alice, to seek assistance from another agent, Bob.
In the first stage, we propose a privacy-aware hypothesis testing-based screening method for Alice to decide on the usefulness of the data from Bob.
We show that Alice can achieve the oracle performance as if the training were from centralized data, both theoretically and numerically.
arXiv Detail & Related papers (2024-05-13T23:24:25Z) - What Hides behind Unfairness? Exploring Dynamics Fairness in Reinforcement Learning [52.51430732904994]
In reinforcement learning problems, agents must consider long-term fairness while maximizing returns.
Recent works have proposed many different types of fairness notions, but how unfairness arises in RL problems remains unclear.
We introduce a novel notion called dynamics fairness, which explicitly captures the inequality stemming from environmental dynamics.
arXiv Detail & Related papers (2024-04-16T22:47:59Z) - Alignment for Honesty [113.42626737461129]
We argue for the importance of alignment for honesty, ensuring that language models proactively refuse to answer questions when they lack knowledge.
This challenge demands comprehensive solutions in terms of metric development, benchmark creation, and training.
We introduce a flexible training framework which is further instantiated by several efficient fine-tuning techniques that emphasize honesty.
arXiv Detail & Related papers (2023-12-12T06:10:42Z) - Adam: Dense Retrieval Distillation with Adaptive Dark Examples [104.01735794498767]
We propose ADAM, a knowledge distillation framework that can better transfer the dark knowledge held in the teacher with Adaptive Dark exAMples.
We conduct experiments on two widely-used benchmarks and verify the effectiveness of our method.
arXiv Detail & Related papers (2022-12-20T12:03:19Z) - Identifying the value of a random variable unambiguously: Quantum versus classical approaches [44.99833362998488]
Quantum resources may provide advantage over their classical counterparts.
We construct such a task based on a game, mediated by Referee and played between Alice and Bob.
We show that if Alice sends limited amount of classical information then the game cannot be won while the quantum analogue of the limited amount of classical information' is sufficient for winning the game.
arXiv Detail & Related papers (2022-11-16T20:28:49Z) - The SAME score: Improved cosine based bias score for word embeddings [63.24247894974291]
We provide a bias definition based on the ideas from the literature and derive novel requirements for bias scores.
We propose a new bias score, SAME, to address the shortcomings of existing bias scores and show empirically that SAME is better suited to quantify biases in word embeddings.
arXiv Detail & Related papers (2022-03-28T09:28:13Z) - Simultaneous Measurement and Entanglement [0.0]
We show how LOCC allows Alice and Bob to distinguish between two product states optimally.
We find that a LOCC is almost always more helpful than a Bell pair for distinguishing product states.
arXiv Detail & Related papers (2022-01-25T23:11:29Z) - Linear Gaussian Quantum State Smoothing: Understanding the optimal
unravelings for Alice to estimate Bob's state [0.0]
Quantum state smoothing is a technique to construct an estimate of the quantum state at a particular time.
The effect of Bob's measurement choice on the effectiveness of Alice's smoothing has been studied in a number of recent papers.
We develop a simple hypothesis that allows one to approximate the optimal measurement choice given Alice's measurement choice.
arXiv Detail & Related papers (2020-08-31T04:04:50Z) - Superadditivity of channel capacity through quantum fields [0.0]
We study the scenario where a sender, Alice, causes information-carrying disturbances in a quantum field.
We find that the channel capacity between Alice and a receiver, Bob, is enhanced by Bob placing detectors not only inside but also outside the causal future of Alice's encoding operation.
arXiv Detail & Related papers (2020-02-11T00:53:34Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.