Assisted Learning for Organizations with Limited Imbalanced Data

• URL: http://arxiv.org/abs/2109.09307v4
• Date: Sat, 2 Mar 2024 10:29:42 GMT
• Title: Assisted Learning for Organizations with Limited Imbalanced Data
• Authors: Cheng Chen, Jiaying Zhou, Jie Ding, Yi Zhou
• Abstract summary: We develop an assisted learning framework for assisting organizations to improve their learning performance. Our framework allows the learner to only occasionally share information with the service provider, but still obtain a model that achieves near-oracle performance.
• Score: 17.34334881241701
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In the era of big data, many big organizations are integrating machine learning into their work pipelines to facilitate data analysis. However, the performance of their trained models is often restricted by limited and imbalanced data available to them. In this work, we develop an assisted learning framework for assisting organizations to improve their learning performance. The organizations have sufficient computation resources but are subject to stringent data-sharing and collaboration policies. Their limited imbalanced data often cause biased inference and sub-optimal decision-making. In assisted learning, an organizational learner purchases assistance service from an external service provider and aims to enhance its model performance within only a few assistance rounds. We develop effective stochastic training algorithms for both assisted deep learning and assisted reinforcement learning. Different from existing distributed algorithms that need to frequently transmit gradients or models, our framework allows the learner to only occasionally share information with the service provider, but still obtain a model that achieves near-oracle performance as if all the data were centralized.

Related papers

• Unlearnable Algorithms for In-context Learning [36.895152458323764]
  In this paper, we focus on efficient unlearning methods for the task adaptation phase of a pretrained large language model. We observe that an LLM's ability to do in-context learning for task adaptation allows for efficient exact unlearning of task adaptation training data. We propose a new holistic measure of unlearning cost which accounts for varying inference costs.
  arXiv  Detail & Related papers  (2024-02-01T16:43:04Z)
• On Efficient Training of Large-Scale Deep Learning Models: A Literature Review [90.87691246153612]
  The field of deep learning has witnessed significant progress, particularly in computer vision (CV), natural language processing (NLP), and speech. The use of large-scale models trained on vast amounts of data holds immense promise for practical applications. With the increasing demands on computational capacity, a comprehensive summarization on acceleration techniques of training deep learning models is still much anticipated.
  arXiv  Detail & Related papers  (2023-04-07T11:13:23Z)
• Federated Pruning: Improving Neural Network Efficiency with Federated Learning [24.36174705715827]
  We propose Federated Pruning to train a reduced model under the federated setting. We explore different pruning schemes and provide empirical evidence of the effectiveness of our methods.
  arXiv  Detail & Related papers  (2022-09-14T00:48:37Z)
• A Survey of Learning on Small Data: Generalization, Optimization, and Challenge [101.27154181792567]
  Learning on small data that approximates the generalization ability of big data is one of the ultimate purposes of AI. This survey follows the active sampling theory under a PAC framework to analyze the generalization error and label complexity of learning on small data. Multiple data applications that may benefit from efficient small data representation are surveyed.
  arXiv  Detail & Related papers  (2022-07-29T02:34:19Z)
• Asynchronous Collaborative Learning Across Data Silos [9.094748832034746]
  We propose a framework to enable asynchronous collaborative training of machine learning models across data silos. This allows data science teams to collaboratively train a machine learning model, without sharing data with one another.
  arXiv  Detail & Related papers  (2022-03-23T18:00:19Z)
• Efficient Device Scheduling with Multi-Job Federated Learning [64.21733164243781]
  We propose a novel multi-job Federated Learning framework to enable the parallel training process of multiple jobs. We propose a reinforcement learning-based method and a Bayesian optimization-based method to schedule devices for multiple jobs while minimizing the cost. Our proposed approaches significantly outperform baseline approaches in terms of training time (up to 8.67 times faster) and accuracy (up to 44.6% higher)
  arXiv  Detail & Related papers  (2021-12-11T08:05:11Z)
• Privacy-Preserving Serverless Edge Learning with Decentralized Small Data [13.254530176359182]
  Distributed training strategies have recently become a promising approach to ensure data privacy when training deep models. This paper extends conventional serverless platforms with serverless edge learning architectures and provides an efficient distributed training framework from the networking perspective.
  arXiv  Detail & Related papers  (2021-11-29T21:04:49Z)
• Understanding the World Through Action [91.3755431537592]
  I will argue that a general, principled, and powerful framework for utilizing unlabeled data can be derived from reinforcement learning. I will discuss how such a procedure is more closely aligned with potential downstream tasks.
  arXiv  Detail & Related papers  (2021-10-24T22:33:52Z)
• Distributed Deep Learning in Open Collaborations [49.240611132653456]
  We propose a novel algorithmic framework designed specifically for collaborative training. We demonstrate the effectiveness of our approach for SwAV and ALBERT pretraining in realistic conditions and achieve performance comparable to traditional setups at a fraction of the cost.
  arXiv  Detail & Related papers  (2021-06-18T16:23:13Z)
• From Distributed Machine Learning to Federated Learning: A Survey [49.7569746460225]
  Federated learning emerges as an efficient approach to exploit distributed data and computing resources. We propose a functional architecture of federated learning systems and a taxonomy of related techniques. We present the distributed training, data communication, and security of FL systems.
  arXiv  Detail & Related papers  (2021-04-29T14:15:11Z)
• Assisted Learning: A Framework for Multi-Organization Learning [16.6491720806269]
  We introduce the Assisted Learning framework for organizations to assist each other in supervised learning tasks without revealing any organization's algorithm, data, or even task. An organization seeks assistance by broadcasting task-specific but nonsensitive statistics and incorporating others' feedback in one or more iterations to eventually improve its predictive performance.
  arXiv  Detail & Related papers  (2020-04-01T16:54:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.