Active Tuning

• URL: http://arxiv.org/abs/2010.03958v2
• Date: Wed, 25 Nov 2020 15:01:40 GMT
• Title: Active Tuning
• Authors: Sebastian Otte, Matthias Karlbauer, Martin V. Butz
• Abstract summary: We introduce Active Tuning, a novel paradigm for optimizing the internal dynamics of neural networks (RNNs) on the fly. In contrast to the conventional sequence-to-imposed mapping scheme, Active Tuning decouples the RNN's recurrent neural activities from the input stream. We demonstrate the effectiveness of Active Tuning on several time series prediction benchmarks.
• Score: 0.5801044612920815
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce Active Tuning, a novel paradigm for optimizing the internal dynamics of recurrent neural networks (RNNs) on the fly. In contrast to the conventional sequence-to-sequence mapping scheme, Active Tuning decouples the RNN's recurrent neural activities from the input stream, using the unfolding temporal gradient signal to tune the internal dynamics into the data stream. As a consequence, the model output depends only on its internal hidden dynamics and the closed-loop feedback of its own predictions; its hidden state is continuously adapted by means of the temporal gradient resulting from backpropagating the discrepancy between the signal observations and the model outputs through time. In this way, Active Tuning infers the signal actively but indirectly based on the originally learned temporal patterns, fitting the most plausible hidden state sequence into the observations. We demonstrate the effectiveness of Active Tuning on several time series prediction benchmarks, including multiple super-imposed sine waves, a chaotic double pendulum, and spatiotemporal wave dynamics. Active Tuning consistently improves the robustness, accuracy, and generalization abilities of all evaluated models. Moreover, networks trained for signal prediction and denoising can be successfully applied to a much larger range of noise conditions with the help of Active Tuning. Thus, given a capable time series predictor, Active Tuning enhances its online signal filtering, denoising, and reconstruction abilities without the need for additional training.

Related papers

• Regime Change Hypothesis: Foundations for Decoupled Dynamics in Neural Network Training [1.0518862318418603]
  In ReLU-based models, the activation pattern induced by a given input determines the piecewise-linear region in which the network behaves affinely.<n>We investigate whether training exhibits a two-timescale behavior: an early stage with substantial changes in activation patterns and a later stage where weight updates predominantly refine the model.
  arXiv  Detail & Related papers  (2026-02-09T07:14:28Z)
• Paradoxical noise preference in RNNs [0.0]
  In recurrent neural networks (RNNs) used to model biological neural networks, noise is typically introduced during training to emulate biological variability and regularize learning.<n>We find that continuous-time recurrent neural networks (CTRNNs) often perform best at a nonzero noise level, specifically, the same level used during training.<n>This noise preference typically arises when noise is injected inside the neural activation function; networks trained with noise injected outside the activation function perform best with zero noise.
  arXiv  Detail & Related papers  (2026-01-08T03:11:51Z)
• Time-Varying Audio Effect Modeling by End-to-End Adversarial Training [0.6688641196358245]
  This paper introduces a Generative Adversarial Network (GAN) framework to model effects using only input-output audio recordings.<n>An initial adversarial phase allows the model to learn the distribution of the modulation behavior without strict phase constraints.<n>A State Prediction Network (SPN) estimates the initial internal states required to synchronize the model with the target.
  arXiv  Detail & Related papers  (2025-12-17T11:04:39Z)
• Langevin Flows for Modeling Neural Latent Dynamics [81.81271685018284]
  We introduce LangevinFlow, a sequential Variational Auto-Encoder where the time evolution of latent variables is governed by the underdamped Langevin equation.<n>Our approach incorporates physical priors -- such as inertia, damping, a learned potential function, and forces -- to represent both autonomous and non-autonomous processes in neural systems.<n>Our method outperforms state-of-the-art baselines on synthetic neural populations generated by a Lorenz attractor.
  arXiv  Detail & Related papers  (2025-07-15T17:57:48Z)
• FDDet: Frequency-Decoupling for Boundary Refinement in Temporal Action Detection [4.015022008487465]
  Large-scale pre-trained video encoders tend to introduce background clutter and irrelevant semantics, leading to context confusion and boundaries. We propose a frequency-aware decoupling network that improves action discriminability by filtering out noisy semantics captured by pre-trained models. Our method achieves state-of-the-art performance on temporal action detection benchmarks.
  arXiv  Detail & Related papers  (2025-04-01T10:57:37Z)
• Allostatic Control of Persistent States in Spiking Neural Networks for perception and computation [79.16635054977068]
  We introduce a novel model for updating perceptual beliefs about the environment by extending the concept of Allostasis to the control of internal representations. In this paper, we focus on an application in numerical cognition, where a bump of activity in an attractor network is used as a spatial numerical representation.
  arXiv  Detail & Related papers  (2025-03-20T12:28:08Z)
• Finding the DeepDream for Time Series: Activation Maximization for Univariate Time Series [10.388704631887496]
  We introduce Sequence Dreaming, a technique that adapts Maxim Activationization to analyze sequential information. We visualize the temporal dynamics and patterns most influential in model decision-making processes.
  arXiv  Detail & Related papers  (2024-08-20T08:09:44Z)
• Signal-SGN: A Spiking Graph Convolutional Network for Skeletal Action Recognition via Learning Temporal-Frequency Dynamics [2.707548544084083]
  Spiking Neural Networks (SNNs) struggle to model skeleton dynamics, leading to suboptimal solutions. We propose Signal-SGN (Spiking Graph Convolutional Network), which utilizes the temporal dimension of skeleton sequences as the spike time steps. Experiments across three large-scale datasets reveal Signal-SGN exceeding state-of-the-art SNN-based methods in accuracy and computational efficiency.
  arXiv  Detail & Related papers  (2024-08-03T07:47:16Z)
• Intensity Profile Projection: A Framework for Continuous-Time Representation Learning for Dynamic Networks [50.2033914945157]
  We present a representation learning framework, Intensity Profile Projection, for continuous-time dynamic network data. The framework consists of three stages: estimating pairwise intensity functions, learning a projection which minimises a notion of intensity reconstruction error. Moreoever, we develop estimation theory providing tight control on the error of any estimated trajectory, indicating that the representations could even be used in quite noise-sensitive follow-on analyses.
  arXiv  Detail & Related papers  (2023-06-09T15:38:25Z)
• How neural networks learn to classify chaotic time series [77.34726150561087]
  We study the inner workings of neural networks trained to classify regular-versus-chaotic time series. We find that the relation between input periodicity and activation periodicity is key for the performance of LKCNN models.
  arXiv  Detail & Related papers  (2023-06-04T08:53:27Z)
• Predicting the temporal dynamics of turbulent channels through deep learning [0.0]
  We aim to assess the capability of neural networks to reproduce the temporal evolution of a minimal turbulent channel flow. Long-short-term-memory (LSTM) networks and a Koopman-based framework (KNF) are trained to predict the temporal dynamics of the minimal-channel-flow modes.
  arXiv  Detail & Related papers  (2022-03-02T09:31:03Z)
• Robust alignment of cross-session recordings of neural population activity by behaviour via unsupervised domain adaptation [1.2617078020344619]
  We introduce a model capable of inferring behaviourally relevant latent dynamics from previously unseen data recorded from the same animal. We show that unsupervised domain adaptation combined with a sequential variational autoencoder, trained on several sessions, can achieve good generalisation to unseen data.
  arXiv  Detail & Related papers  (2022-02-12T22:17:30Z)
• Deep Impulse Responses: Estimating and Parameterizing Filters with Deep Networks [76.830358429947]
  Impulse response estimation in high noise and in-the-wild settings is a challenging problem. We propose a novel framework for parameterizing and estimating impulse responses based on recent advances in neural representation learning.
  arXiv  Detail & Related papers  (2022-02-07T18:57:23Z)
• Deep Explicit Duration Switching Models for Time Series [84.33678003781908]
  We propose a flexible model that is capable of identifying both state- and time-dependent switching dynamics. State-dependent switching is enabled by a recurrent state-to-switch connection. An explicit duration count variable is used to improve the time-dependent switching behavior.
  arXiv  Detail & Related papers  (2021-10-26T17:35:21Z)
• Unfolding recurrence by Green's functions for optimized reservoir computing [3.7823923040445995]
  Cortical networks are strongly recurrent, and neurons have intrinsic temporal dynamics. This sets them apart from deep feed-forward networks. We present a solvable recurrent network model that links to feed forward networks.
  arXiv  Detail & Related papers  (2020-10-13T09:17:10Z)
• Inferring, Predicting, and Denoising Causal Wave Dynamics [3.9407250051441403]
  The DISTributed Artificial neural Network Architecture (DISTANA) is a generative, recurrent graph convolution neural network. We show that DISTANA is very well-suited to denoise data streams, given that re-occurring patterns are observed. It produces stable and accurate closed-loop predictions even over hundreds of time steps.
  arXiv  Detail & Related papers  (2020-09-19T08:33:53Z)
• A Prospective Study on Sequence-Driven Temporal Sampling and Ego-Motion Compensation for Action Recognition in the EPIC-Kitchens Dataset [68.8204255655161]
  Action recognition is one of the top-challenging research fields in computer vision. ego-motion recorded sequences have become of important relevance. The proposed method aims to cope with it by estimating this ego-motion or camera motion.
  arXiv  Detail & Related papers  (2020-08-26T14:44:45Z)
• Liquid Time-constant Networks [117.57116214802504]
  We introduce a new class of time-continuous recurrent neural network models. Instead of declaring a learning system's dynamics by implicit nonlinearities, we construct networks of linear first-order dynamical systems. These neural networks exhibit stable and bounded behavior, yield superior expressivity within the family of neural ordinary differential equations.
  arXiv  Detail & Related papers  (2020-06-08T09:53:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.