Ensemble deep learning: A review

• URL: http://arxiv.org/abs/2104.02395v1
• Date: Tue, 6 Apr 2021 09:56:29 GMT
• Title: Ensemble deep learning: A review
• Authors: M.A. Ganaie (1) and Minghui Hu (2) and M. Tanveer*(1) and P.N. Suganthan*(2) (* Corresponding Author (1) Department of Mathematics, Indian Institute of Technology Indore, Simrol, Indore, 453552, India (2) School of Electrical & Electronic Engineering, Nanyang Technological University, Singapore)
• Abstract summary: Ensemble learning combines several individual models to obtain better generalization performance. Deep ensemble learning models combine the advantages of both the deep learning models as well as the ensemble learning. This paper reviews the state-of-art deep ensemble models and hence serves as an extensive summary for the researchers.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Ensemble learning combines several individual models to obtain better generalization performance. Currently, deep learning models with multilayer processing architecture is showing better performance as compared to the shallow or traditional classification models. Deep ensemble learning models combine the advantages of both the deep learning models as well as the ensemble learning such that the final model has better generalization performance. This paper reviews the state-of-art deep ensemble models and hence serves as an extensive summary for the researchers. The ensemble models are broadly categorised into ensemble models like bagging, boosting and stacking, negative correlation based deep ensemble models, explicit/implicit ensembles, homogeneous /heterogeneous ensemble, decision fusion strategies, unsupervised, semi-supervised, reinforcement learning and online/incremental, multilabel based deep ensemble models. Application of deep ensemble models in different domains is also briefly discussed. Finally, we conclude this paper with some future recommendations and research directions.

Related papers

• A Collaborative Ensemble Framework for CTR Prediction [73.59868761656317]
  We propose a novel framework, Collaborative Ensemble Training Network (CETNet), to leverage multiple distinct models. Unlike naive model scaling, our approach emphasizes diversity and collaboration through collaborative learning. We validate our framework on three public datasets and a large-scale industrial dataset from Meta.
  arXiv  Detail & Related papers  (2024-11-20T20:38:56Z)
• What Matters for Model Merging at Scale? [94.26607564817786]
  Model merging aims to combine multiple expert models into a more capable single model. Previous studies have primarily focused on merging a few small models. This study systematically evaluates the utility of model merging at scale.
  arXiv  Detail & Related papers  (2024-10-04T17:17:19Z)
• Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications and Opportunities [89.40778301238642]
  Model merging is an efficient empowerment technique in the machine learning community. There is a significant gap in the literature regarding a systematic and thorough review of these techniques.
  arXiv  Detail & Related papers  (2024-08-14T16:58:48Z)
• Modern Neighborhood Components Analysis: A Deep Tabular Baseline Two Decades Later [59.88557193062348]
  We revisit the classic Neighborhood Component Analysis (NCA), designed to learn a linear projection that captures semantic similarities between instances. We find that minor modifications, such as adjustments to the learning objectives and the integration of deep learning architectures, significantly enhance NCA's performance. We also introduce a neighbor sampling strategy that improves both the efficiency and predictive accuracy of our proposed ModernNCA.
  arXiv  Detail & Related papers  (2024-07-03T16:38:57Z)
• Learnable & Interpretable Model Combination in Dynamic Systems Modeling [0.0]
  We discuss which types of models are usually combined and propose a model interface that is capable of expressing a variety of mixed equation based models. We propose a new wildcard topology, that is capable of describing the generic connection between two combined models in an easy to interpret fashion. The contributions of this paper are highlighted at a proof of concept: Different connection topologies between two models are learned, interpreted and compared.
  arXiv  Detail & Related papers  (2024-06-12T11:17:11Z)
• FusionBench: A Comprehensive Benchmark of Deep Model Fusion [78.80920533793595]
  Deep model fusion is a technique that unifies the predictions or parameters of several deep neural networks into a single model. FusionBench is the first comprehensive benchmark dedicated to deep model fusion.
  arXiv  Detail & Related papers  (2024-06-05T13:54:28Z)
• A Review of Sparse Expert Models in Deep Learning [23.721204843236006]
  Sparse expert models are a thirty-year old concept re-emerging as a popular architecture in deep learning. We review the concept of sparse expert models, provide a basic description of the common algorithms, and contextualize the advances in the deep learning era.
  arXiv  Detail & Related papers  (2022-09-04T18:00:29Z)
• Self-paced ensemble learning for speech and audio classification [19.39192082485334]
  We propose a self-paced ensemble learning scheme in which models learn from each other over several iterations. During the self-paced learning process, our ensemble also gains knowledge about the target domain. Our empirical results indicate that SPEL significantly outperforms the baseline ensemble models.
  arXiv  Detail & Related papers  (2021-03-22T16:34:06Z)
• Model Complexity of Deep Learning: A Survey [79.20117679251766]
  We conduct a systematic overview of the latest studies on model complexity in deep learning. We review the existing studies on those two categories along four important factors, including model framework, model size, optimization process and data complexity.
  arXiv  Detail & Related papers  (2021-03-08T22:39:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.