S2RMs: Spatially Structured Recurrent Modules

• URL: http://arxiv.org/abs/2007.06533v1
• Date: Mon, 13 Jul 2020 17:44:30 GMT
• Title: S2RMs: Spatially Structured Recurrent Modules
• Authors: Nasim Rahaman, Anirudh Goyal, Muhammad Waleed Gondal, Manuel Wuthrich, Stefan Bauer, Yash Sharma, Yoshua Bengio, Bernhard Sch\"olkopf
• Abstract summary: We take a step towards exploiting dynamic structure that are capable of simultaneously exploiting both modular andtemporal structures. We find our models to be robust to the number of available views and better capable of generalization to novel tasks without additional training.
• Score: 105.0377129434636
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Capturing the structure of a data-generating process by means of appropriate inductive biases can help in learning models that generalize well and are robust to changes in the input distribution. While methods that harness spatial and temporal structures find broad application, recent work has demonstrated the potential of models that leverage sparse and modular structure using an ensemble of sparingly interacting modules. In this work, we take a step towards dynamic models that are capable of simultaneously exploiting both modular and spatiotemporal structures. We accomplish this by abstracting the modeled dynamical system as a collection of autonomous but sparsely interacting sub-systems. The sub-systems interact according to a topology that is learned, but also informed by the spatial structure of the underlying real-world system. This results in a class of models that are well suited for modeling the dynamics of systems that only offer local views into their state, along with corresponding spatial locations of those views. On the tasks of video prediction from cropped frames and multi-agent world modeling from partial observations in the challenging Starcraft2 domain, we find our models to be more robust to the number of available views and better capable of generalization to novel tasks without additional training, even when compared against strong baselines that perform equally well or better on the training distribution.

Related papers

• Topological Deep Learning with State-Space Models: A Mamba Approach for Simplicial Complexes [4.787059527893628]
  We propose a novel architecture designed to operate with simplicial complexes, utilizing the Mamba state-space model as its backbone. Our approach generates sequences for the nodes based on the neighboring cells, enabling direct communication between all higher-order structures, regardless of their rank.
  arXiv  Detail & Related papers  (2024-09-18T14:49:25Z)
• Learnable & Interpretable Model Combination in Dynamic Systems Modeling [0.0]
  We discuss which types of models are usually combined and propose a model interface that is capable of expressing a variety of mixed equation based models. We propose a new wildcard topology, that is capable of describing the generic connection between two combined models in an easy to interpret fashion. The contributions of this paper are highlighted at a proof of concept: Different connection topologies between two models are learned, interpreted and compared.
  arXiv  Detail & Related papers  (2024-06-12T11:17:11Z)
• Modular Deep Learning [120.36599591042908]
  Transfer learning has recently become the dominant paradigm of machine learning. It remains unclear how to develop models that specialise towards multiple tasks without incurring negative interference. Modular deep learning has emerged as a promising solution to these challenges.
  arXiv  Detail & Related papers  (2023-02-22T18:11:25Z)
• Learning Modular Simulations for Homogeneous Systems [23.355189771765644]
  We present a modular simulation framework for modeling homogeneous multibody dynamical systems. An arbitrary number of modules can be combined to simulate systems of a variety of coupling topologies. We show that our models can be transferred to new system configurations lower with data requirement and training effort, compared to those trained from scratch.
  arXiv  Detail & Related papers  (2022-10-28T17:48:01Z)
• ModLaNets: Learning Generalisable Dynamics via Modularity and Physical Inductive Bias [14.474273671369584]
  We propose a structural neural network framework with modularity and physical inductive bias. This framework models the energy of each element using modularity and then construct the target dynamical system via Lagrangian mechanics. We examine our framework for modelling double-pendulum or three-body systems with small training datasets.
  arXiv  Detail & Related papers  (2022-06-24T14:54:25Z)
• Model LEGO: Creating Models Like Disassembling and Assembling Building Blocks [53.09649785009528]
  In this paper, we explore a paradigm that does not require training to obtain new models. Similar to the birth of CNN inspired by receptive fields in the biological visual system, we propose Model Disassembling and Assembling. For model assembling, we present the alignment padding strategy and parameter scaling strategy to construct a new model tailored for a specific task.
  arXiv  Detail & Related papers  (2022-03-25T05:27:28Z)
• Low-Rank Constraints for Fast Inference in Structured Models [110.38427965904266]
  This work demonstrates a simple approach to reduce the computational and memory complexity of a large class of structured models. Experiments with neural parameterized structured models for language modeling, polyphonic music modeling, unsupervised grammar induction, and video modeling show that our approach matches the accuracy of standard models at large state spaces.
  arXiv  Detail & Related papers  (2022-01-08T00:47:50Z)
• GEM: Group Enhanced Model for Learning Dynamical Control Systems [78.56159072162103]
  We build effective dynamical models that are amenable to sample-based learning. We show that learning the dynamics on a Lie algebra vector space is more effective than learning a direct state transition model. This work sheds light on a connection between learning of dynamics and Lie group properties, which opens doors for new research directions.
  arXiv  Detail & Related papers  (2021-04-07T01:08:18Z)
• Learning Stable Deep Dynamics Models [91.90131512825504]
  We propose an approach for learning dynamical systems that are guaranteed to be stable over the entire state space. We show that such learning systems are able to model simple dynamical systems and can be combined with additional deep generative models to learn complex dynamics.
  arXiv  Detail & Related papers  (2020-01-17T00:04:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.