Unified Implementations of Recurrent Neural Networks in Multiple Deep Learning Frameworks

• URL: http://arxiv.org/abs/2510.21252v1
• Date: Fri, 24 Oct 2025 08:35:33 GMT
• Title: Unified Implementations of Recurrent Neural Networks in Multiple Deep Learning Frameworks
• Authors: Francesco Martinuzzi,
• Abstract summary: torchrecurrent, RecurrentLayers.jl, and LuxRecurrentLayers.jl offer a consistent framework for constructing and extending RNN models.<n>All packages are available under the MIT license and actively maintained on GitHub.
• Score: 0.5187177298223502
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recurrent neural networks (RNNs) are a cornerstone of sequence modeling across various scientific and industrial applications. Owing to their versatility, numerous RNN variants have been proposed over the past decade, aiming to improve the modeling of long-term dependencies and to address challenges such as vanishing and exploding gradients. However, no central library is available to test these variations, and reimplementing diverse architectures can be time-consuming and error-prone, limiting reproducibility and exploration. Here, we introduce three open-source libraries in Julia and Python that centralize numerous recurrent cell implementations and higher-level recurrent architectures. torchrecurrent, RecurrentLayers.jl, and LuxRecurrentLayers.jl offer a consistent framework for constructing and extending RNN models, providing built-in mechanisms for customization and experimentation. All packages are available under the MIT license and actively maintained on GitHub.

Related papers

• $\ exttt{lrnnx}$: A library for Linear RNNs [0.0]
  Linear recurrent neural networks (LRNNs) provide a structured approach to sequence modeling that bridges classical linear dynamical systems and modern deep learning, offering both expressive power and theoretical guarantees on stability and trainability.<n>Existing implementations are fragmented across different software frameworks, often rely on framework-specific optimizations, and in some cases require custom kernels or lack publicly available code altogether.<n>We introduce $textttlrn$, a unified software library that implements several modern LRNN architectures under a common interface.
  arXiv  Detail & Related papers  (2026-02-09T15:48:48Z)
• A Retrieval-Augmented Generation Approach to Extracting Algorithmic Logic from Neural Networks [48.83701310501069]
  We introduce NN-RAG, a retrieval-augmented generation system that converts large, heterogeneous PyTorchs into a searchable library of validated neural modules.<n>Applying to 19 major repositories, the pipeline extracted 1,289 candidate blocks, validated 941 (73.0%), and demonstrated that over 80% are structurally unique.
  arXiv  Detail & Related papers  (2025-12-03T23:28:30Z)
• An Efficient and Mixed Heterogeneous Model for Image Restoration [71.85124734060665]
  Current mainstream approaches are based on three architectural paradigms: CNNs, Transformers, and Mambas.<n>We propose RestorMixer, an efficient and general-purpose IR model based on mixed-architecture fusion.
  arXiv  Detail & Related papers  (2025-04-15T08:19:12Z)
• NNsight and NDIF: Democratizing Access to Open-Weight Foundation Model Internals [58.83169560132308]
  We introduce NNsight and NDIF, technologies that work in tandem to enable scientific study of the representations and computations learned by very large neural networks.
  arXiv  Detail & Related papers  (2024-07-18T17:59:01Z)
• TorchDEQ: A Library for Deep Equilibrium Models [72.65236284030894]
  We present TorchDEQ, an out-of-the-box library that allows users to define, train, and infer using DEQs over multiple domains with minimal code and best practices. We build a DEQ Zoo'' that supports six published implicit models across different domains. We have substantially improved the performance, training stability, and efficiency of DEQs on ten datasets across all six projects in the DEQ Zoo.
  arXiv  Detail & Related papers  (2023-10-28T06:16:10Z)
• cito: An R package for training neural networks using torch [0.0]
  'cito' is a user-friendly R package for deep learning (DL) applications. It allows specifying DNNs in the familiar formula syntax used by many R packages. 'cito' includes many user-friendly functions for model plotting and analysis.
  arXiv  Detail & Related papers  (2023-03-16T18:54:20Z)
• Seasonal Encoder-Decoder Architecture for Forecasting [1.9188864062289432]
  We propose a novel RNN architecture capturing (stochastic) seasonal correlations intelligently. It is motivated from the well-known encoder-decoder (ED) architecture and multiplicative seasonal auto-regressive model. It can be employed on single or multiple sequence data.
  arXiv  Detail & Related papers  (2022-07-08T20:06:45Z)
• Deep Neural Networks pruning via the Structured Perspective Regularization [5.061851539114448]
  In Machine Learning, Artificial Neural Networks (ANNs) are a very powerful tool, broadly used in many applications. One of the most popular compression approaches is emphpruning, whereby entire elements of the ANN (links, nodes, channels, ldots) and the corresponding weights are deleted. Since the nature of the problem is inherently (what elements to prune and what not), we propose a new pruning method based on Operational Research tools.
  arXiv  Detail & Related papers  (2022-06-28T14:58:51Z)
• TyXe: Pyro-based Bayesian neural nets for Pytorch [12.343312954353639]
  We introduce TyXe, a Bayesian neural network library built on top of Pytorch and Pyro. Our leading design principle is to cleanly separate architecture, prior, inference and likelihood specification. In contrast to existing packages TyXe does not implement any layer classes, and instead relies on architectures defined in generic Pytorch code.
  arXiv  Detail & Related papers  (2021-10-01T09:04:26Z)
• D-DARTS: Distributed Differentiable Architecture Search [75.12821786565318]
  Differentiable ARchiTecture Search (DARTS) is one of the most trending Neural Architecture Search (NAS) methods. We propose D-DARTS, a novel solution that addresses this problem by nesting several neural networks at cell-level.
  arXiv  Detail & Related papers  (2021-08-20T09:07:01Z)
• Incremental Training of a Recurrent Neural Network Exploiting a Multi-Scale Dynamic Memory [79.42778415729475]
  We propose a novel incrementally trained recurrent architecture targeting explicitly multi-scale learning. We show how to extend the architecture of a simple RNN by separating its hidden state into different modules. We discuss a training algorithm where new modules are iteratively added to the model to learn progressively longer dependencies.
  arXiv  Detail & Related papers  (2020-06-29T08:35:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.