Autoencoder based Randomized Learning of Feedforward Neural Networks for Regression

• URL: http://arxiv.org/abs/2107.01711v1
• Date: Sun, 4 Jul 2021 19:07:39 GMT
• Title: Autoencoder based Randomized Learning of Feedforward Neural Networks for Regression
• Authors: Grzegorz Dudek
• Abstract summary: gradient-based learning suffers from many drawbacks making the training process ineffective and time-consuming. Alternative randomized learning does not use gradients but selects hidden node parameters randomly. A recently proposed method uses autoencoders for unsupervised parameter learning.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Feedforward neural networks are widely used as universal predictive models to fit data distribution. Common gradient-based learning, however, suffers from many drawbacks making the training process ineffective and time-consuming. Alternative randomized learning does not use gradients but selects hidden node parameters randomly. This makes the training process extremely fast. However, the problem in randomized learning is how to determine the random parameters. A recently proposed method uses autoencoders for unsupervised parameter learning. This method showed superior performance on classification tasks. In this work, we apply this method to regression problems, and, finding that it has some drawbacks, we show how to improve it. We propose a learning method of autoencoders that controls the produced random weights. We also propose how to determine the biases of hidden nodes. We empirically compare autoencoder based learning with other randomized learning methods proposed recently for regression and find that despite the proposed improvement of the autoencoder based learning, it does not outperform its competitors in fitting accuracy. Moreover, the method is much more complex than its competitors.

Related papers

• Enhancing Consistency and Mitigating Bias: A Data Replay Approach for Incremental Learning [100.7407460674153]
  Deep learning systems are prone to catastrophic forgetting when learning from a sequence of tasks. To mitigate the problem, a line of methods propose to replay the data of experienced tasks when learning new tasks. However, it is not expected in practice considering the memory constraint or data privacy issue. As a replacement, data-free data replay methods are proposed by inverting samples from the classification model.
  arXiv  Detail & Related papers  (2024-01-12T12:51:12Z)
• Learning to Optimize Permutation Flow Shop Scheduling via Graph-based Imitation Learning [70.65666982566655]
  Permutation flow shop scheduling (PFSS) is widely used in manufacturing systems. We propose to train the model via expert-driven imitation learning, which accelerates convergence more stably and accurately. Our model's network parameters are reduced to only 37% of theirs, and the solution gap of our model towards the expert solutions decreases from 6.8% to 1.3% on average.
  arXiv  Detail & Related papers  (2022-10-31T09:46:26Z)
• Refining neural network predictions using background knowledge [68.35246878394702]
  We show we can use logical background knowledge in learning system to compensate for a lack of labeled training data. We introduce differentiable refinement functions that find a corrected prediction close to the original prediction. This algorithm finds optimal refinements on complex SAT formulas in significantly fewer iterations and frequently finds solutions where gradient descent can not.
  arXiv  Detail & Related papers  (2022-06-10T10:17:59Z)
• Invariance Learning in Deep Neural Networks with Differentiable Laplace Approximations [76.82124752950148]
  We develop a convenient gradient-based method for selecting the data augmentation. We use a differentiable Kronecker-factored Laplace approximation to the marginal likelihood as our objective.
  arXiv  Detail & Related papers  (2022-02-22T02:51:11Z)
• Learning Fast Sample Re-weighting Without Reward Data [41.92662851886547]
  This paper presents a novel learning-based fast sample re-weighting (FSR) method that does not require additional reward data. Our experiments show the proposed method achieves competitive results compared to state of the arts on label noise and long-tailed recognition.
  arXiv  Detail & Related papers  (2021-09-07T17:30:56Z)
• A concise method for feature selection via normalized frequencies [0.0]
  In this paper, a concise method is proposed for universal feature selection. The proposed method uses a fusion of the filter method and the wrapper method, rather than a combination of them. The evaluation results show that the proposed method outperformed several state-of-the-art related works in terms of accuracy, precision, recall, F-score and AUC.
  arXiv  Detail & Related papers  (2021-06-10T15:29:54Z)
• On the Reproducibility of Neural Network Predictions [52.47827424679645]
  We study the problem of churn, identify factors that cause it, and propose two simple means of mitigating it. We first demonstrate that churn is indeed an issue, even for standard image classification tasks. We propose using emphminimum entropy regularizers to increase prediction confidences. We present empirical results showing the effectiveness of both techniques in reducing churn while improving the accuracy of the underlying model.
  arXiv  Detail & Related papers  (2021-02-05T18:51:01Z)
• Cross-Validation and Uncertainty Determination for Randomized Neural Networks with Applications to Mobile Sensors [0.0]
  Extreme learning machines provide an attractive and efficient method for supervised learning under limited computing ressources and green machine learning. Results are discussed about supervised learning with such networks and regression methods in terms of consistency and bounds for the generalization and prediction error.
  arXiv  Detail & Related papers  (2021-01-06T12:28:06Z)
• Category-Learning with Context-Augmented Autoencoder [63.05016513788047]
  Finding an interpretable non-redundant representation of real-world data is one of the key problems in Machine Learning. We propose a novel method of using data augmentations when training autoencoders. We train a Variational Autoencoder in such a way, that it makes transformation outcome predictable by auxiliary network.
  arXiv  Detail & Related papers  (2020-10-10T14:04:44Z)
• Deep Ordinal Regression with Label Diversity [19.89482062012177]
  We propose that using several discrete data representations simultaneously can improve neural network learning. Our approach is end-to-end differentiable and can be added as a simple extension to conventional learning methods.
  arXiv  Detail & Related papers  (2020-06-29T08:23:43Z)
• Auto-Ensemble: An Adaptive Learning Rate Scheduling based Deep Learning Model Ensembling [11.324407834445422]
  This paper proposes Auto-Ensemble (AE) to collect checkpoints of deep learning model and ensemble them automatically. The advantage of this method is to make the model converge to various local optima by scheduling the learning rate in once training.
  arXiv  Detail & Related papers  (2020-03-25T08:17:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.