Collaborative Group Learning

• URL: http://arxiv.org/abs/2009.07712v4
• Date: Mon, 22 Feb 2021 04:57:36 GMT
• Title: Collaborative Group Learning
• Authors: Shaoxiong Feng, Hongshen Chen, Xuancheng Ren, Zhuoye Ding, Kan Li, Xu Sun
• Abstract summary: Collaborative learning has successfully applied knowledge transfer to guide a pool of small student networks towards robust local minima. Previous approaches typically struggle with drastically aggravated student homogenization when the number of students rises. We propose Collaborative Group Learning, an efficient framework that aims to diversify the feature representation and conduct an effective regularization.
• Score: 42.31194030839819
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Collaborative learning has successfully applied knowledge transfer to guide a pool of small student networks towards robust local minima. However, previous approaches typically struggle with drastically aggravated student homogenization when the number of students rises. In this paper, we propose Collaborative Group Learning, an efficient framework that aims to diversify the feature representation and conduct an effective regularization. Intuitively, similar to the human group study mechanism, we induce students to learn and exchange different parts of course knowledge as collaborative groups. First, each student is established by randomly routing on a modular neural network, which facilitates flexible knowledge communication between students due to random levels of representation sharing and branching. Second, to resist the student homogenization, students first compose diverse feature sets by exploiting the inductive bias from sub-sets of training data, and then aggregate and distill different complementary knowledge by imitating a random sub-group of students at each time step. Overall, the above mechanisms are beneficial for maximizing the student population to further improve the model generalization without sacrificing computational efficiency. Empirical evaluations on both image and text tasks indicate that our method significantly outperforms various state-of-the-art collaborative approaches whilst enhancing computational efficiency.

Related papers

• RIGL: A Unified Reciprocal Approach for Tracing the Independent and Group Learning Processes [22.379764500005503]
  We propose RIGL, a unified Reciprocal model to trace knowledge states at both the individual and group levels. In this paper, we introduce a time frame-aware reciprocal embedding module to concurrently model both student and group response interactions. We design a relation-guided temporal attentive network, comprised of dynamic graph modeling coupled with a temporal self-attention mechanism.
  arXiv  Detail & Related papers  (2024-06-18T10:16:18Z)
• Graph Enhanced Reinforcement Learning for Effective Group Formation in Collaborative Problem Solving [3.392758494801288]
  This study addresses the challenge of forming effective groups in collaborative problem-solving environments. We propose a novel approach leveraging graph theory and reinforcement learning.
  arXiv  Detail & Related papers  (2024-03-15T04:04:40Z)
• Decentralized and Lifelong-Adaptive Multi-Agent Collaborative Learning [57.652899266553035]
  Decentralized and lifelong-adaptive multi-agent collaborative learning aims to enhance collaboration among multiple agents without a central server. We propose DeLAMA, a decentralized multi-agent lifelong collaborative learning algorithm with dynamic collaboration graphs.
  arXiv  Detail & Related papers  (2024-03-11T09:21:11Z)
• Joint Training of Deep Ensembles Fails Due to Learner Collusion [61.557412796012535]
  Ensembles of machine learning models have been well established as a powerful method of improving performance over a single model. Traditionally, ensembling algorithms train their base learners independently or sequentially with the goal of optimizing their joint performance. We show that directly minimizing the loss of the ensemble appears to rarely be applied in practice.
  arXiv  Detail & Related papers  (2023-01-26T18:58:07Z)
• Distantly-Supervised Named Entity Recognition with Adaptive Teacher Learning and Fine-grained Student Ensemble [56.705249154629264]
  Self-training teacher-student frameworks are proposed to improve the robustness of NER models. In this paper, we propose an adaptive teacher learning comprised of two teacher-student networks. Fine-grained student ensemble updates each fragment of the teacher model with a temporal moving average of the corresponding fragment of the student, which enhances consistent predictions on each model fragment against noise.
  arXiv  Detail & Related papers  (2022-12-13T12:14:09Z)
• Mitigating Biases in Student Performance Prediction via Attention-Based Personalized Federated Learning [7.040747348755578]
  Traditional learning-based approaches to student modeling generalize poorly to underrepresented student groups due to biases in data availability. We propose a methodology for predicting student performance from their online learning activities that optimize inference accuracy over different demographic groups such as race and gender.
  arXiv  Detail & Related papers  (2022-08-02T00:22:20Z)
• Towards Explainable Student Group Collaboration Assessment Models Using Temporal Representations of Individual Student Roles [12.945344702592557]
  We propose using simple temporal-CNN deep-learning models to assess student group collaboration. We check the applicability of dynamically changing feature representations for student group collaboration assessment. We also use Grad-CAM visualizations to better understand and interpret the important temporal indices that led to the deep-learning model's decision.
  arXiv  Detail & Related papers  (2021-06-17T16:00:08Z)
• MultiFair: Multi-Group Fairness in Machine Learning [52.24956510371455]
  We study multi-group fairness in machine learning (MultiFair) We propose a generic end-to-end algorithmic framework to solve it. Our proposed framework is generalizable to many different settings.
  arXiv  Detail & Related papers  (2021-05-24T02:30:22Z)
• Multi-View Feature Representation for Dialogue Generation with Bidirectional Distillation [22.14228918338769]
  We propose a novel training framework, where the learning of general knowledge is more in line with the idea of reaching consensus. Our framework effectively improves the model generalization without sacrificing training efficiency.
  arXiv  Detail & Related papers  (2021-02-22T05:23:34Z)
• Task-Feature Collaborative Learning with Application to Personalized Attribute Prediction [166.87111665908333]
  We propose a novel multi-task learning method called Task-Feature Collaborative Learning (TFCL) Specifically, we first propose a base model with a heterogeneous block-diagonal structure regularizer to leverage the collaborative grouping of features and tasks. As a practical extension, we extend the base model by allowing overlapping features and differentiating the hard tasks.
  arXiv  Detail & Related papers  (2020-04-29T02:32:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.