Collaboratively Learning Linear Models with Structured Missing Data
- URL: http://arxiv.org/abs/2307.11947v1
- Date: Sat, 22 Jul 2023 00:07:10 GMT
- Title: Collaboratively Learning Linear Models with Structured Missing Data
- Authors: Chen Cheng, Gary Cheng, John Duchi
- Abstract summary: We study the problem of collaboratively least squares estimates for $magents.
Our goal is to determine how to coordinate the agents in learning to produce the best estimator for each agent.
- Score: 3.4376560669160394
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We study the problem of collaboratively learning least squares estimates for
$m$ agents. Each agent observes a different subset of the
features$\unicode{x2013}$e.g., containing data collected from sensors of
varying resolution. Our goal is to determine how to coordinate the agents in
order to produce the best estimator for each agent. We propose a distributed,
semi-supervised algorithm Collab, consisting of three steps: local training,
aggregation, and distribution. Our procedure does not require communicating the
labeled data, making it communication efficient and useful in settings where
the labeled data is inaccessible. Despite this handicap, our procedure is
nearly asymptotically local minimax optimal$\unicode{x2013}$even among
estimators allowed to communicate the labeled data such as imputation methods.
We test our method on real and synthetic data.
Related papers
- Inverse Entropic Optimal Transport Solves Semi-supervised Learning via Data Likelihood Maximization [65.8915778873691]
conditional distributions is a central problem in machine learning.
We propose a new learning paradigm that integrates both paired and unpaired data.
Our approach also connects intriguingly with inverse entropic optimal transport (OT)
arXiv Detail & Related papers (2024-10-03T16:12:59Z) - Data Sharing for Mean Estimation Among Heterogeneous Strategic Agents [11.371461065112422]
We study a collaborative learning problem where $m$ agents estimate a vector $muinmathbbRd$ by collecting samples from normal distributions.
Instead of working on their own, agents can collect data that is cheap to them, and share it with others in exchange for data that is expensive or even inaccessible to them.
arXiv Detail & Related papers (2024-07-20T17:45:40Z) - Numerical Data Imputation for Multimodal Data Sets: A Probabilistic
Nearest-Neighbor Kernel Density Approach [2.750124853532831]
We introduce a data imputation method combining nearest neighbor estimation ($k$NN) and density estimation with Gaussian kernels (KDE)
We show that our method can cope with complex original data structure, yields lower data imputation errors, and provides probabilistic estimates with higher likelihood than current methods.
arXiv Detail & Related papers (2023-06-29T12:55:58Z) - Bi-level Alignment for Cross-Domain Crowd Counting [113.78303285148041]
Current methods rely on external data for training an auxiliary task or apply an expensive coarse-to-fine estimation.
We develop a new adversarial learning based method, which is simple and efficient to apply.
We evaluate our approach on five real-world crowd counting benchmarks, where we outperform existing approaches by a large margin.
arXiv Detail & Related papers (2022-05-12T02:23:25Z) - Leveraging Ensembles and Self-Supervised Learning for Fully-Unsupervised
Person Re-Identification and Text Authorship Attribution [77.85461690214551]
Learning from fully-unlabeled data is challenging in Multimedia Forensics problems, such as Person Re-Identification and Text Authorship Attribution.
Recent self-supervised learning methods have shown to be effective when dealing with fully-unlabeled data in cases where the underlying classes have significant semantic differences.
We propose a strategy to tackle Person Re-Identification and Text Authorship Attribution by enabling learning from unlabeled data even when samples from different classes are not prominently diverse.
arXiv Detail & Related papers (2022-02-07T13:08:11Z) - Leveraging Unlabeled Data to Predict Out-of-Distribution Performance [63.740181251997306]
Real-world machine learning deployments are characterized by mismatches between the source (training) and target (test) distributions.
In this work, we investigate methods for predicting the target domain accuracy using only labeled source data and unlabeled target data.
We propose Average Thresholded Confidence (ATC), a practical method that learns a threshold on the model's confidence, predicting accuracy as the fraction of unlabeled examples.
arXiv Detail & Related papers (2022-01-11T23:01:12Z) - Dealing with Distribution Mismatch in Semi-supervised Deep Learning for
Covid-19 Detection Using Chest X-ray Images: A Novel Approach Using Feature
Densities [0.6882042556551609]
Semi-supervised deep learning is an attractive alternative to large labelled datasets.
In real-world usage settings, an unlabelled dataset might present a different distribution than the labelled dataset.
This results in a distribution mismatch between the unlabelled and labelled datasets.
arXiv Detail & Related papers (2021-08-17T00:35:43Z) - Achieving differential privacy for $k$-nearest neighbors based outlier
detection by data partitioning [0.3437656066916039]
We develop a differentially private ($epsilon$-DP) approach for $k$-NN based outlier detection.
We propose a method for $k$-NN based outlier detection by separating the procedure into a fitting step on reference inlier data and then apply the outlier classifier to new data.
Our approach yields nearly optimal performance on real-world data with varying dimensions when compared to the non-private versions of $k$-NN.
arXiv Detail & Related papers (2021-04-16T07:35:26Z) - How to distribute data across tasks for meta-learning? [59.608652082495624]
We show that the optimal number of data points per task depends on the budget, but it converges to a unique constant value for large budgets.
Our results suggest a simple and efficient procedure for data collection.
arXiv Detail & Related papers (2021-03-15T15:38:47Z) - Learning while Respecting Privacy and Robustness to Distributional
Uncertainties and Adversarial Data [66.78671826743884]
The distributionally robust optimization framework is considered for training a parametric model.
The objective is to endow the trained model with robustness against adversarially manipulated input data.
Proposed algorithms offer robustness with little overhead.
arXiv Detail & Related papers (2020-07-07T18:25:25Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.