Perceptual adjustment queries and an inverted measurement paradigm for
low-rank metric learning
- URL: http://arxiv.org/abs/2309.04626v1
- Date: Fri, 8 Sep 2023 22:36:33 GMT
- Title: Perceptual adjustment queries and an inverted measurement paradigm for
low-rank metric learning
- Authors: Austin Xu, Andrew D. McRae, Jingyan Wang, Mark A. Davenport, Ashwin
Pananjady
- Abstract summary: We introduce a new type of query mechanism for collecting human feedback, called the perceptual adjustment query ( PAQ)
Being both informative and cognitively lightweight, the PAQ adopts an inverted measurement scheme, and combines advantages from both cardinal and ordinal queries.
We develop a two-stage estimator for metric learning from PAQs, and provide sample complexity guarantees for this estimator.
- Score: 22.7492766005919
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We introduce a new type of query mechanism for collecting human feedback,
called the perceptual adjustment query ( PAQ). Being both informative and
cognitively lightweight, the PAQ adopts an inverted measurement scheme, and
combines advantages from both cardinal and ordinal queries. We showcase the PAQ
in the metric learning problem, where we collect PAQ measurements to learn an
unknown Mahalanobis distance. This gives rise to a high-dimensional, low-rank
matrix estimation problem to which standard matrix estimators cannot be
applied. Consequently, we develop a two-stage estimator for metric learning
from PAQs, and provide sample complexity guarantees for this estimator. We
present numerical simulations demonstrating the performance of the estimator
and its notable properties.
Related papers
- Bayesian Prediction-Powered Inference [62.2436697657307]
Prediction-powered inference (PPI) is a method that improves statistical estimates based on limited human-labeled data.
We propose a framework for PPI based on Bayesian inference that allows researchers to develop new task-appropriate PPI methods easily.
arXiv Detail & Related papers (2024-05-09T18:08:58Z) - Query Performance Prediction using Relevance Judgments Generated by Large Language Models [53.97064615557883]
We propose a QPP framework using automatically generated relevance judgments (QPP-GenRE)
QPP-GenRE decomposes QPP into independent subtasks of predicting relevance of each item in a ranked list to a given query.
This allows us to predict any IR evaluation measure using the generated relevance judgments as pseudo-labels.
arXiv Detail & Related papers (2024-04-01T09:33:05Z) - Leveraging Variational Autoencoders for Parameterized MMSE Estimation [10.141454378473972]
We propose a variational autoencoder-based framework for parameterizing a conditional linear minimum mean squared error estimator.
The derived estimator is shown to approximate the minimum mean squared error estimator by utilizing the variational autoencoder as a generative prior for the estimation problem.
We conduct a rigorous analysis by bounding the difference between the proposed and the minimum mean squared error estimator.
arXiv Detail & Related papers (2023-07-11T15:41:34Z) - Towards Clear Expectations for Uncertainty Estimation [64.20262246029286]
Uncertainty Quantification (UQ) is crucial to achieve trustworthy Machine Learning (ML)
Most UQ methods suffer from disparate and inconsistent evaluation protocols.
This opinion paper offers a new perspective by specifying those requirements through five downstream tasks.
arXiv Detail & Related papers (2022-07-27T07:50:57Z) - QAFactEval: Improved QA-Based Factual Consistency Evaluation for
Summarization [116.56171113972944]
We show that carefully choosing the components of a QA-based metric is critical to performance.
Our solution improves upon the best-performing entailment-based metric and achieves state-of-the-art performance.
arXiv Detail & Related papers (2021-12-16T00:38:35Z) - Adaptive estimation of quantum observables [4.567122178196833]
We introduce a measurement scheme that adaptively modifies the estimator based on previously obtained data.
Our algorithm, which we call AEQuO, continuously monitors both the estimated average and the associated error of the considered observable.
We test our protocol on chemistry Hamiltonians, for which AEQuO provides error estimates that improve on all state-of-the-art methods.
arXiv Detail & Related papers (2021-10-28T17:49:16Z) - Frequentist Parameter Estimation with Supervised Learning [0.0]
We use regression to infer a machine-learned point estimate of an unknown parameter.
When the number of training measurements are large, this is identical to the well-known maximum-likelihood estimator (MLE)
We show that the machine-learned estimator inherits the desirable properties of the MLE, up to a limit imposed by the resolution of the training grid.
arXiv Detail & Related papers (2021-05-26T02:24:25Z) - Nonparametric Estimation of the Fisher Information and Its Applications [82.00720226775964]
This paper considers the problem of estimation of the Fisher information for location from a random sample of size $n$.
An estimator proposed by Bhattacharya is revisited and improved convergence rates are derived.
A new estimator, termed a clipped estimator, is proposed.
arXiv Detail & Related papers (2020-05-07T17:21:56Z) - Machine learning for causal inference: on the use of cross-fit
estimators [77.34726150561087]
Doubly-robust cross-fit estimators have been proposed to yield better statistical properties.
We conducted a simulation study to assess the performance of several estimators for the average causal effect (ACE)
When used with machine learning, the doubly-robust cross-fit estimators substantially outperformed all of the other estimators in terms of bias, variance, and confidence interval coverage.
arXiv Detail & Related papers (2020-04-21T23:09:55Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.