The Effect of Training Schedules on Morphological Robustness and Generalization

• URL: http://arxiv.org/abs/2407.13965v1
• Date: Fri, 19 Jul 2024 00:55:31 GMT
• Title: The Effect of Training Schedules on Morphological Robustness and Generalization
• Authors: Edoardo Barba, Anil Yaman, Giovanni Iacca,
• Abstract summary: Robustness and generalizability are key properties of artificial neural network (ANN)-based controllers. In this paper, we focus on morphological robustness and generalizability concerned with finding an ANN-based controller that can provide sufficient performance on a range of physical variations.
• Score: 11.068618445120508
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Robustness and generalizability are the key properties of artificial neural network (ANN)-based controllers for maintaining a reliable performance in case of changes. It is demonstrated that exposing the ANNs to variations during training processes can improve their robustness and generalization capabilities. However, the way in which this variation is introduced can have a significant impact. In this paper, we define various training schedules to specify how these variations are introduced during an evolutionary learning process. In particular, we focus on morphological robustness and generalizability concerned with finding an ANN-based controller that can provide sufficient performance on a range of physical variations. Then, we perform an extensive analysis of the effect of these training schedules on morphological generalization. Furthermore, we formalize the process of training sample selection (i.e., morphological variations) to improve generalization as a reinforcement learning problem. Overall, our results provide deeper insights into the role of variability and the ways of enhancing the generalization property of evolved ANN-based controllers.

Related papers

• REACT: Revealing Evolutionary Action Consequence Trajectories for Interpretable Reinforcement Learning [7.889696505137217]
  We propose Revealing Evolutionary Action Consequence Trajectories (REACT) to enhance the interpretability of Reinforcement Learning (RL) In contrast to the prevalent practice of RL models based on their optimal behavior learned during training, we posit that considering a range of edge-case trajectories provides a more comprehensive understanding of their inherent behavior. Our results highlight its effectiveness in revealing nuanced aspects of RL models' behavior beyond optimal performance, thereby contributing to improved interpretability.
  arXiv  Detail & Related papers  (2024-04-04T10:56:30Z)
• Revisiting Dynamic Evaluation: Online Adaptation for Large Language Models [88.47454470043552]
  We consider the problem of online fine tuning the parameters of a language model at test time, also known as dynamic evaluation. Online adaptation turns parameters into temporally changing states and provides a form of context-length extension with memory in weights.
  arXiv  Detail & Related papers  (2024-03-03T14:03:48Z)
• Task adaption by biologically inspired stochastic comodulation [8.59194778459436]
  We show that fine-tuning convolutional networks by gain modulation improves on deterministic gain modulation. Our results suggest that comodulation representations can enhance learning efficiency and performance in multi-task learning.
  arXiv  Detail & Related papers  (2023-11-25T15:21:03Z)
• Evolving generalist controllers to handle a wide range of morphological variations [1.4425878137951238]
  The study of robustness and generalizability of artificial neural networks (ANNs) has remained limited. Unexpected morphological or environmental changes during operation can risk failure if the ANN controllers are unable to handle these changes. This paper proposes an algorithm that aims to enhance the robustness and generalizability of the controllers.
  arXiv  Detail & Related papers  (2023-09-18T23:06:19Z)
• NormAUG: Normalization-guided Augmentation for Domain Generalization [60.159546669021346]
  We propose a simple yet effective method called NormAUG (Normalization-guided Augmentation) for deep learning. Our method introduces diverse information at the feature level and improves the generalization of the main path. In the test stage, we leverage an ensemble strategy to combine the predictions from the auxiliary path of our model, further boosting performance.
  arXiv  Detail & Related papers  (2023-07-25T13:35:45Z)
• TANGOS: Regularizing Tabular Neural Networks through Gradient Orthogonalization and Specialization [69.80141512683254]
  We introduce Tabular Neural Gradient Orthogonalization and gradient (TANGOS) TANGOS is a novel framework for regularization in the tabular setting built on latent unit attributions. We demonstrate that our approach can lead to improved out-of-sample generalization performance, outperforming other popular regularization methods.
  arXiv  Detail & Related papers  (2023-03-09T18:57:13Z)
• Latent Variable Representation for Reinforcement Learning [131.03944557979725]
  It remains unclear theoretically and empirically how latent variable models may facilitate learning, planning, and exploration to improve the sample efficiency of model-based reinforcement learning. We provide a representation view of the latent variable models for state-action value functions, which allows both tractable variational learning algorithm and effective implementation of the optimism/pessimism principle. In particular, we propose a computationally efficient planning algorithm with UCB exploration by incorporating kernel embeddings of latent variable models.
  arXiv  Detail & Related papers  (2022-12-17T00:26:31Z)
• The Role of Morphological Variation in Evolutionary Robotics: Maximizing Performance and Robustness [0.0]
  We introduce a method that permits us to measure the impact of morphological variations. We analyze the relation between the amplitude of variations, the modality with which they are introduced, and the performance and robustness of evolving agents. Our results show that morphological variations permit generating solutions which perform better in varying and non-varying conditions.
  arXiv  Detail & Related papers  (2022-08-04T17:58:15Z)
• Data Augmentation vs. Equivariant Networks: A Theory of Generalization on Dynamics Forecasting [24.363954435050264]
  Exploiting symmetry in dynamical systems is a powerful way to improve the generalization of deep learning. Data augmentation and equivariant networks are two major approaches to injecting symmetry into learning. We derive the generalization bounds for data augmentation and equivariant networks, characterizing their effect on learning in a unified framework.
  arXiv  Detail & Related papers  (2022-06-19T17:00:12Z)
• Trajectory-wise Multiple Choice Learning for Dynamics Generalization in Reinforcement Learning [137.39196753245105]
  We present a new model-based reinforcement learning algorithm that learns a multi-headed dynamics model for dynamics generalization. We incorporate context learning, which encodes dynamics-specific information from past experiences into the context latent vector. Our method exhibits superior zero-shot generalization performance across a variety of control tasks, compared to state-of-the-art RL methods.
  arXiv  Detail & Related papers  (2020-10-26T03:20:42Z)
• Dynamics Generalization via Information Bottleneck in Deep Reinforcement Learning [90.93035276307239]
  We propose an information theoretic regularization objective and an annealing-based optimization method to achieve better generalization ability in RL agents. We demonstrate the extreme generalization benefits of our approach in different domains ranging from maze navigation to robotic tasks. This work provides a principled way to improve generalization in RL by gradually removing information that is redundant for task-solving.
  arXiv  Detail & Related papers  (2020-08-03T02:24:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.