A Critical Analysis of the Theoretical Framework of the Extreme Learning Machine
- URL: http://arxiv.org/abs/2406.17427v1
- Date: Tue, 25 Jun 2024 10:06:07 GMT
- Title: A Critical Analysis of the Theoretical Framework of the Extreme Learning Machine
- Authors: Irina Perfilievaa, Nicolas Madrid, Manuel Ojeda-Aciego, Piotr Artiemjew, Agnieszka Niemczynowicz,
- Abstract summary: We refute the proofs of two main statements and a dataset that provides a counterexample to the ELM learning algorithm.
We provide alternative statements of the foundations, which justify the efficiency of ELM in some theoretical cases.
- Score: 1.9503475832401784
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Despite the number of successful applications of the Extreme Learning Machine (ELM), we show that its underlying foundational principles do not have a rigorous mathematical justification. Specifically, we refute the proofs of two main statements, and we also create a dataset that provides a counterexample to the ELM learning algorithm and explain its design, which leads to many such counterexamples. Finally, we provide alternative statements of the foundations, which justify the efficiency of ELM in some theoretical cases.
Related papers
- ConceptMath: A Bilingual Concept-wise Benchmark for Measuring
Mathematical Reasoning of Large Language Models [67.32868432113587]
This paper introduces ConceptMath, a fine-grained benchmark that evaluates concept-wise mathematical reasoning of Large Language Models (LLMs)
Unlike traditional benchmarks that evaluate general mathematical reasoning with an average accuracy, ConceptMath systematically organizes math problems under a hierarchy of math concepts.
arXiv Detail & Related papers (2024-02-22T16:06:49Z) - A Principled Framework for Knowledge-enhanced Large Language Model [58.1536118111993]
Large Language Models (LLMs) are versatile, yet they often falter in tasks requiring deep and reliable reasoning.
This paper introduces a rigorously designed framework for creating LLMs that effectively anchor knowledge and employ a closed-loop reasoning process.
arXiv Detail & Related papers (2023-11-18T18:10:02Z) - Towards the Theory of Unsupervised Federated Learning: Non-asymptotic Analysis of Federated EM Algorithms [13.857921574409362]
We introduce a federated gradient EM algorithm (FedGrEM) designed for the unsupervised learning of mixture models.
We present a comprehensive finite-sample theory that holds for general mixture models.
We then apply this general theory on specific statistical models to characterize the explicit estimation error of model parameters and mixture proportions.
arXiv Detail & Related papers (2023-10-23T19:53:36Z) - Enriching Disentanglement: From Logical Definitions to Quantitative Metrics [59.12308034729482]
Disentangling the explanatory factors in complex data is a promising approach for generalizable and data-efficient representation learning.
We establish a theoretical connection between logical definitions of disentanglement and quantitative metrics using topos theory and enriched category theory.
We empirically demonstrate the effectiveness of the proposed metrics by isolating different aspects of disentangled representations.
arXiv Detail & Related papers (2023-05-19T08:22:23Z) - Synergies between Disentanglement and Sparsity: Generalization and
Identifiability in Multi-Task Learning [79.83792914684985]
We prove a new identifiability result that provides conditions under which maximally sparse base-predictors yield disentangled representations.
Motivated by this theoretical result, we propose a practical approach to learn disentangled representations based on a sparsity-promoting bi-level optimization problem.
arXiv Detail & Related papers (2022-11-26T21:02:09Z) - Representation Theory for Geometric Quantum Machine Learning [0.0]
Recent advances in classical machine learning have shown that creating models with inductive biases encoding the symmetries of a problem can greatly improve performance.
Geometric Quantum Machine Learning (GQML) will play a crucial role in developing problem-specific and quantum-aware models.
We present an introduction to representation theory tools from the optics of quantum learning, driven by key examples involving discrete and continuous groups.
arXiv Detail & Related papers (2022-10-14T17:25:36Z) - Making Linear MDPs Practical via Contrastive Representation Learning [101.75885788118131]
It is common to address the curse of dimensionality in Markov decision processes (MDPs) by exploiting low-rank representations.
We consider an alternative definition of linear MDPs that automatically ensures normalization while allowing efficient representation learning.
We demonstrate superior performance over existing state-of-the-art model-based and model-free algorithms on several benchmarks.
arXiv Detail & Related papers (2022-07-14T18:18:02Z) - Efficient Performance Bounds for Primal-Dual Reinforcement Learning from
Demonstrations [1.0609815608017066]
We consider large-scale Markov decision processes with an unknown cost function and address the problem of learning a policy from a finite set of expert demonstrations.
Existing inverse reinforcement learning methods come with strong theoretical guarantees, but are computationally expensive.
We introduce a novel bilinear saddle-point framework using Lagrangian duality to bridge the gap between theory and practice.
arXiv Detail & Related papers (2021-12-28T05:47:24Z) - Optimization and Sampling Under Continuous Symmetry: Examples and Lie
Theory [26.555110725656963]
We show examples of Lieant's theorem, Lie groups, Lie algebras, and the Harish-Chandra--Itzyintegrals formulas.
We then present an introduction to optimization theory -- an indispensable mathematical toolkit for capturing continuous symmetries.
arXiv Detail & Related papers (2021-09-02T16:44:44Z) - Training a First-Order Theorem Prover from Synthetic Data [50.23600875138756]
A major challenge in applying machine learning to automated theorem proving is the scarcity of training data.
We propose an approach that relies on training purely with synthetically generated theorems, without any human data aside from axioms.
Our neural prover outperforms the state-of-the-art E-prover on this synthetic data in both time and search steps.
arXiv Detail & Related papers (2021-03-05T17:01:34Z) - Learning to Prove from Synthetic Theorems [41.74768503409581]
A major challenge in applying machine learning to automated theorem proving is the scarcity of training data.
We propose an approach that relies on training with synthetic theorems, generated from a set of axioms.
We show that such theorems can be used to train an automated prover and that the learned prover transfers successfully to human-generated theorems.
arXiv Detail & Related papers (2020-06-19T17:48:09Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.