Prediction and Generalisation over Directed Actions by Grid Cells
- URL: http://arxiv.org/abs/2006.03355v2
- Date: Tue, 23 Mar 2021 02:05:30 GMT
- Title: Prediction and Generalisation over Directed Actions by Grid Cells
- Authors: Changmin Yu, Timothy E.J. Behrens and Neil Burgess
- Abstract summary: Knowing how directed actions generalise to new situations is key to rapid generalisation.
Recent work has proposed that neural grid codes provide an efficient representation of the state space.
We show that a single set of eigenvectors can support predictions over arbitrary directed actions via action-specific eigenvalues.
- Score: 6.7141720056953895
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Knowing how the effects of directed actions generalise to new situations
(e.g. moving North, South, East and West, or turning left, right, etc.) is key
to rapid generalisation across new situations. Markovian tasks can be
characterised by a state space and a transition matrix and recent work has
proposed that neural grid codes provide an efficient representation of the
state space, as eigenvectors of a transition matrix reflecting diffusion across
states, that allows efficient prediction of future state distributions. Here we
extend the eigenbasis prediction model, utilising tools from Fourier analysis,
to prediction over arbitrary translation-invariant directed transition
structures (i.e. displacement and diffusion), showing that a single set of
eigenvectors can support predictions over arbitrary directed actions via
action-specific eigenvalues. We show how to define a "sense of direction" to
combine actions to reach a target state (ignoring task-specific deviations from
translation-invariance), and demonstrate that adding the Fourier
representations to a deep Q network aids policy learning in continuous control
tasks. We show the equivalence between the generalised prediction framework and
traditional models of grid cell firing driven by self-motion to perform path
integration, either using oscillatory interference (via Fourier components as
velocity-controlled oscillators) or continuous attractor networks (via analysis
of the update dynamics). We thus provide a unifying framework for the role of
the grid system in predictive planning, sense of direction and path
integration: supporting generalisable inference over directed actions across
different tasks.
Related papers
- Dita: Scaling Diffusion Transformer for Generalist Vision-Language-Action Policy [56.424032454461695]
We present Dita, a scalable framework that leverages Transformer architectures to directly denoise continuous action sequences.
Dita employs in-context conditioning -- enabling fine-grained alignment between denoised actions and raw visual tokens from historical observations.
Dita effectively integrates cross-embodiment datasets across diverse camera perspectives, observation scenes, tasks, and action spaces.
arXiv Detail & Related papers (2025-03-25T15:19:56Z) - Words in Motion: Extracting Interpretable Control Vectors for Motion Transformers [1.1510009152620668]
Transformer-based models generate hidden states that are difficult to interpret.
We analyze hidden states and modify them at inference, with a focus on motion forecasting.
arXiv Detail & Related papers (2024-06-17T15:07:55Z) - AMP: Autoregressive Motion Prediction Revisited with Next Token Prediction for Autonomous Driving [59.94343412438211]
We introduce the GPT style next token motion prediction into motion prediction.
Different from language data which is composed of homogeneous units -words, the elements in the driving scene could have complex spatial-temporal and semantic relations.
We propose to adopt three factorized attention modules with different neighbors for information aggregation and different position encoding styles to capture their relations.
arXiv Detail & Related papers (2024-03-20T06:22:37Z) - Towards Generalizable and Interpretable Motion Prediction: A Deep
Variational Bayes Approach [54.429396802848224]
This paper proposes an interpretable generative model for motion prediction with robust generalizability to out-of-distribution cases.
For interpretability, the model achieves the target-driven motion prediction by estimating the spatial distribution of long-term destinations.
Experiments on motion prediction datasets validate that the fitted model can be interpretable and generalizable.
arXiv Detail & Related papers (2024-03-10T04:16:04Z) - Out-of-Distribution Generalized Dynamic Graph Neural Network with
Disentangled Intervention and Invariance Promotion [61.751257172868186]
Dynamic graph neural networks (DyGNNs) have demonstrated powerful predictive abilities by exploiting graph and temporal dynamics.
Existing DyGNNs fail to handle distribution shifts, which naturally exist in dynamic graphs.
arXiv Detail & Related papers (2023-11-24T02:42:42Z) - Equivariant Map and Agent Geometry for Autonomous Driving Motion
Prediction [4.096893535332546]
This research introduces a groundbreaking solution by employing EqMotion, a theoretically geometric equivariant and interaction invariant motion prediction model for particles and humans.
By applying these technologies, our model is able to achieve high prediction accuracy while maintain a lightweight design and efficient data utilization.
arXiv Detail & Related papers (2023-10-21T07:08:44Z) - Control-Aware Prediction Objectives for Autonomous Driving [78.19515972466063]
We present control-aware prediction objectives (CAPOs) to evaluate the downstream effect of predictions on control without requiring the planner be differentiable.
We propose two types of importance weights that weight the predictive likelihood: one using an attention model between agents, and another based on control variation when exchanging predicted trajectories for ground truth trajectories.
arXiv Detail & Related papers (2022-04-28T07:37:21Z) - End-to-End Trajectory Distribution Prediction Based on Occupancy Grid
Maps [29.67295706224478]
In this paper, we aim to forecast a future trajectory distribution of a moving agent in the real world, given the social scene images and historical trajectories.
We learn the distribution with symmetric cross-entropy using occupancy grid maps as an explicit and scene-compliant approximation to the ground-truth distribution.
In experiments, our method achieves state-of-the-art performance on the Stanford Drone dataset and Intersection Drone dataset.
arXiv Detail & Related papers (2022-03-31T09:24:32Z) - Stochastic Trajectory Prediction via Motion Indeterminacy Diffusion [88.45326906116165]
We present a new framework to formulate the trajectory prediction task as a reverse process of motion indeterminacy diffusion (MID)
We encode the history behavior information and the social interactions as a state embedding and devise a Transformer-based diffusion model to capture the temporal dependencies of trajectories.
Experiments on the human trajectory prediction benchmarks including the Stanford Drone and ETH/UCY datasets demonstrate the superiority of our method.
arXiv Detail & Related papers (2022-03-25T16:59:08Z) - Flowformer: Linearizing Transformers with Conservation Flows [77.25101425464773]
We linearize Transformers free from specific inductive biases based on the flow network theory.
By respectively conserving the incoming flow of sinks for source competition and the outgoing flow of sources for sink allocation, Flow-Attention inherently generates informative attentions.
arXiv Detail & Related papers (2022-02-13T08:44:10Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.