Learning predictable and robust neural representations by straightening image sequences

• URL: http://arxiv.org/abs/2411.01777v1
• Date: Mon, 04 Nov 2024 03:58:09 GMT
• Title: Learning predictable and robust neural representations by straightening image sequences
• Authors: Xueyan Niu, Cristina Savin, Eero P. Simoncelli,
• Abstract summary: We develop a self-supervised learning (SSL) objective that explicitly quantifies and promotes straightening. We demonstrate the power of this objective in training deep feedforward neural networks on smoothly-rendered synthetic image sequences.
• Score: 16.504807843249196
• License:
• Abstract: Prediction is a fundamental capability of all living organisms, and has been proposed as an objective for learning sensory representations. Recent work demonstrates that in primate visual systems, prediction is facilitated by neural representations that follow straighter temporal trajectories than their initial photoreceptor encoding, which allows for prediction by linear extrapolation. Inspired by these experimental findings, we develop a self-supervised learning (SSL) objective that explicitly quantifies and promotes straightening. We demonstrate the power of this objective in training deep feedforward neural networks on smoothly-rendered synthetic image sequences that mimic commonly-occurring properties of natural videos. The learned model contains neural embeddings that are predictive, but also factorize the geometric, photometric, and semantic attributes of objects. The representations also prove more robust to noise and adversarial attacks compared to previous SSL methods that optimize for invariance to random augmentations. Moreover, these beneficial properties can be transferred to other training procedures by using the straightening objective as a regularizer, suggesting a broader utility for straightening as a principle for robust unsupervised learning.

Related papers

• Deep Learning Through A Telescoping Lens: A Simple Model Provides Empirical Insights On Grokking, Gradient Boosting & Beyond [61.18736646013446]
  In pursuit of a deeper understanding of its surprising behaviors, we investigate the utility of a simple yet accurate model of a trained neural network. Across three case studies, we illustrate how it can be applied to derive new empirical insights on a diverse range of prominent phenomena.
  arXiv  Detail & Related papers  (2024-10-31T22:54:34Z)
• A distributional simplicity bias in the learning dynamics of transformers [50.91742043564049]
  We show that transformers, trained on natural language data, also display a simplicity bias. Specifically, they sequentially learn many-body interactions among input tokens, reaching a saturation point in the prediction error for low-degree interactions. This approach opens up the possibilities of studying how interactions of different orders in the data affect learning, in natural language processing and beyond.
  arXiv  Detail & Related papers  (2024-10-25T15:39:34Z)
• Brain-like representational straightening of natural movies in robust feedforward neural networks [2.8749107965043286]
  Representational straightening refers to a decrease in curvature of visual feature representations of a sequence of frames taken from natural movies. We show robustness to noise in the input image can produce representational straightening in feedforward neural networks.
  arXiv  Detail & Related papers  (2023-08-26T13:04:36Z)
• A polar prediction model for learning to represent visual transformations [10.857320773825357]
  We propose a self-supervised representation-learning framework that exploits the regularities of natural videos to compute accurate predictions. When trained on natural video datasets, our framework achieves better prediction performance than traditional motion compensation. Our framework offers a principled framework for understanding how the visual system represents sensory inputs in a form that simplifies temporal prediction.
  arXiv  Detail & Related papers  (2023-03-06T19:00:59Z)
• Understanding Self-Predictive Learning for Reinforcement Learning [61.62067048348786]
  We study the learning dynamics of self-predictive learning for reinforcement learning. We propose a novel self-predictive algorithm that learns two representations simultaneously.
  arXiv  Detail & Related papers  (2022-12-06T20:43:37Z)
• Feature visualization for convolutional neural network models trained on neuroimaging data [0.0]
  We show for the first time results using feature visualization of convolutional neural networks (CNNs) We have trained CNNs for different tasks including sex classification and artificial lesion classification based on structural magnetic resonance imaging (MRI) data. The resulting images reveal the learned concepts of the artificial lesions, including their shapes, but remain hard to interpret for abstract features in the sex classification task.
  arXiv  Detail & Related papers  (2022-03-24T15:24:38Z)
• Data-driven emergence of convolutional structure in neural networks [83.4920717252233]
  We show how fully-connected neural networks solving a discrimination task can learn a convolutional structure directly from their inputs. By carefully designing data models, we show that the emergence of this pattern is triggered by the non-Gaussian, higher-order local structure of the inputs.
  arXiv  Detail & Related papers  (2022-02-01T17:11:13Z)
• Learning Personal Representations from fMRIby Predicting Neurofeedback Performance [52.77024349608834]
  We present a deep neural network method for learning a personal representation for individuals performing a self neuromodulation task, guided by functional MRI (fMRI) The representation is learned by a self-supervised recurrent neural network, that predicts the Amygdala activity in the next fMRI frame given recent fMRI frames and is conditioned on the learned individual representation.
  arXiv  Detail & Related papers  (2021-12-06T10:16:54Z)
• Neuronal Learning Analysis using Cycle-Consistent Adversarial Networks [4.874780144224057]
  We use a variant of deep generative models called - CycleGAN, to learn the unknown mapping between pre- and post-learning neural activities. We develop an end-to-end pipeline to preprocess, train and evaluate calcium fluorescence signals, and a procedure to interpret the resulting deep learning models.
  arXiv  Detail & Related papers  (2021-11-25T13:24:19Z)
• PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learning [109.84770951839289]
  We present PredRNN, a new recurrent network for learning visual dynamics from historical context. We show that our approach obtains highly competitive results on three standard datasets.
  arXiv  Detail & Related papers  (2021-03-17T08:28:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.