Modular Deep Recurrent Neural Network: Application to Quadrotors

• URL: http://arxiv.org/abs/2609.04339v1
• Date: Thu, 03 Sep 2026 18:06:46 GMT
• Title: Modular Deep Recurrent Neural Network: Application to Quadrotors
• Abstract summary: The capability of the RNN to learn and model high-order dynamics and nonlinearities is significantly improved.<n>The problem of vanishing/exploding gradient in space for a multilayer RNN is also alleviated using feedforward connections.
• Score: 13.551450190727033
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A modular deep Recurrent Neural Network (RNN) is introduced to facilitate the process of deploying various architectures of RNNs, and to automatically compute derivatives for gradient-based learning methods. The modularity leads to a set of new architectures, one of which includes feedforward inter-layer connections. By adding feedforward inter-layer connections in a multi-layer RNN, it is observed that the capability of the RNN to learn and model high-order dynamics and nonlinearities is significantly improved. The problem of vanishing/exploding gradient in space for a multilayer RNN is also alleviated using feedforward connections. These results are demonstrated using a quadrotor case study, for which a model of the altitude dynamics is learned with our particular network structure, while existing methods are unable to generalize as quickly or at all.

Related papers

• Scalable Learning in Structured Recurrent Spiking Neural Networks without Backpropagation [6.572000338549628]
  Spiking Neural Networks (SNNs) provide a promising framework for energy-efficient and biologically grounded computation.<n>We propose a structured multi-layer recurrent SNN architecture composed of locally dense recurrent layers augmented with sparse small-world long-range projections to a readout population.<n>The long-range connectivity is largely fixed, preserving routing efficiency and hardware scalability, while synaptic adaptation is performed using strictly local plasticity mechanisms.
  arXiv  Detail & Related papers  (2026-05-01T04:45:21Z)
• Unveiling Hidden Convexity in Deep Learning: a Sparse Signal Processing Perspective [51.908606610432194]
  Deep neural networks (DNNs) have achieved remarkable success across diverse machine learning tasks.<n>Recent research has uncovered several convexities in the loss of certain NN architectures.
  arXiv  Detail & Related papers  (2026-03-25T01:34:04Z)
• Surrogate-Assisted Evolution for Efficient Multi-branch Connection Design in Deep Neural Networks [3.113634696452565]
  State-of-the-art Deep Neural Networks (DNNs) often incorporate multi-branch connections.<n>We introduce a novel approach based on Linear Genetic Programming (LGP) to encode multi-branch (MB) connections within DNNs.<n>We scale their use from dozens or hundreds of sample points to thousands, aligning with the demands of complex DNNs.
  arXiv  Detail & Related papers  (2025-06-25T14:18:17Z)
• Residual Kolmogorov-Arnold Network for Enhanced Deep Learning [0.9399249626168465]
  Deep convolutional neural networks (CNNs) can be difficult to optimize and costly to train due to hundreds of layers within the network depth.<n>We introduce a "plug-in" module, called Residual Kolmogorov-Arnold Network (RKAN)<n>RKAN offers consistent improvements over baseline models in different vision tasks.
  arXiv  Detail & Related papers  (2024-10-07T21:12:32Z)
• Deep Architecture Connectivity Matters for Its Convergence: A Fine-Grained Analysis [94.64007376939735]
  We theoretically characterize the impact of connectivity patterns on the convergence of deep neural networks (DNNs) under gradient descent training. We show that by a simple filtration on "unpromising" connectivity patterns, we can trim down the number of models to evaluate.
  arXiv  Detail & Related papers  (2022-05-11T17:43:54Z)
• Hybrid-Layers Neural Network Architectures for Modeling the Self-Interference in Full-Duplex Systems [23.55330151898652]
  Full analysis (FD) systems provide simultaneous transmission of information over frequency resources. This article proposes two novel hybrid-layers neural network (NN) architectures to cancel localized with low complexity. The proposed NNs exploit, in a novel manner, a combination of hidden layers (e.g., dense) in order to model the SI with lower computational complexity than the state-of-the-art NN-based cancelers.
  arXiv  Detail & Related papers  (2021-10-18T14:18:56Z)
• SIRe-Networks: Skip Connections over Interlaced Multi-Task Learning and Residual Connections for Structure Preserving Object Classification [28.02302915971059]
  In this paper, we introduce an interlaced multi-task learning strategy, defined SIRe, to reduce the vanishing gradient in relation to the object classification task. The presented methodology directly improves a convolutional neural network (CNN) by enforcing the input image structure preservation through auto-encoders. To validate the presented methodology, a simple CNN and various implementations of famous networks are extended via the SIRe strategy and extensively tested on the CIFAR100 dataset.
  arXiv  Detail & Related papers  (2021-10-06T13:54:49Z)
• A novel Deep Neural Network architecture for non-linear system identification [78.69776924618505]
  We present a novel Deep Neural Network (DNN) architecture for non-linear system identification. Inspired by fading memory systems, we introduce inductive bias (on the architecture) and regularization (on the loss function) This architecture allows for automatic complexity selection based solely on available data.
  arXiv  Detail & Related papers  (2021-06-06T10:06:07Z)
• Coupled Oscillatory Recurrent Neural Network (coRNN): An accurate and (gradient) stable architecture for learning long time dependencies [15.2292571922932]
  We propose a novel architecture for recurrent neural networks. Our proposed RNN is based on a time-discretization of a system of second-order ordinary differential equations. Experiments show that the proposed RNN is comparable in performance to the state of the art on a variety of benchmarks.
  arXiv  Detail & Related papers  (2020-10-02T12:35:04Z)
• Accurate and Lightweight Image Super-Resolution with Model-Guided Deep Unfolding Network [63.69237156340457]
  We present and advocate an explainable approach toward SISR named model-guided deep unfolding network (MoG-DUN) MoG-DUN is accurate (producing fewer aliasing artifacts), computationally efficient (with reduced model parameters), and versatile (capable of handling multiple degradations) The superiority of the proposed MoG-DUN method to existing state-of-theart image methods including RCAN, SRDNF, and SRFBN is substantiated by extensive experiments on several popular datasets and various degradation scenarios.
  arXiv  Detail & Related papers  (2020-09-14T08:23:37Z)
• Modeling from Features: a Mean-field Framework for Over-parameterized Deep Neural Networks [54.27962244835622]
  This paper proposes a new mean-field framework for over- parameterized deep neural networks (DNNs) In this framework, a DNN is represented by probability measures and functions over its features in the continuous limit. We illustrate the framework via the standard DNN and the Residual Network (Res-Net) architectures.
  arXiv  Detail & Related papers  (2020-07-03T01:37:16Z)
• Progressive Tandem Learning for Pattern Recognition with Deep Spiking Neural Networks [80.15411508088522]
  Spiking neural networks (SNNs) have shown advantages over traditional artificial neural networks (ANNs) for low latency and high computational efficiency. We propose a novel ANN-to-SNN conversion and layer-wise learning framework for rapid and efficient pattern recognition.
  arXiv  Detail & Related papers  (2020-07-02T15:38:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.