Layerwise complexity-matched learning yields an improved model of cortical area V2

• URL: http://arxiv.org/abs/2312.11436v3
• Date: Thu, 18 Jul 2024 23:41:24 GMT
• Title: Layerwise complexity-matched learning yields an improved model of cortical area V2
• Authors: Nikhil Parthasarathy, Olivier J. Hénaff, Eero P. Simoncelli,
• Abstract summary: Deep neural networks trained end-to-end for object recognition approach human capabilities. We develop a self-supervised training methodology that operates independently on successive layers. We show that our model is better aligned with selectivity properties and neural activity in primate area V2.
• Score: 12.861402235256207
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Human ability to recognize complex visual patterns arises through transformations performed by successive areas in the ventral visual cortex. Deep neural networks trained end-to-end for object recognition approach human capabilities, and offer the best descriptions to date of neural responses in the late stages of the hierarchy. But these networks provide a poor account of the early stages, compared to traditional hand-engineered models, or models optimized for coding efficiency or prediction. Moreover, the gradient backpropagation used in end-to-end learning is generally considered to be biologically implausible. Here, we overcome both of these limitations by developing a bottom-up self-supervised training methodology that operates independently on successive layers. Specifically, we maximize feature similarity between pairs of locally-deformed natural image patches, while decorrelating features across patches sampled from other images. Crucially, the deformation amplitudes are adjusted proportionally to receptive field sizes in each layer, thus matching the task complexity to the capacity at each stage of processing. In comparison with architecture-matched versions of previous models, we demonstrate that our layerwise complexity-matched learning (LCL) formulation produces a two-stage model (LCL-V2) that is better aligned with selectivity properties and neural activity in primate area V2. We demonstrate that the complexity-matched learning paradigm is responsible for much of the emergence of the improved biological alignment. Finally, when the two-stage model is used as a fixed front-end for a deep network trained to perform object recognition, the resultant model (LCL-V2Net) is significantly better than standard end-to-end self-supervised, supervised, and adversarially-trained models in terms of generalization to out-of-distribution tasks and alignment with human behavior.

Related papers

• Analysing the Behaviour of Tree-Based Neural Networks in Regression Tasks [3.912345988363511]
  This paper endeavours to decode the behaviour of tree-based neural network models in the context of regression challenges. We extend the application of established models--tree-based CNNs, Code2Vec, and Transformer-based methods--to predict the execution time of source code by parsing it to an AST. Our proposed dual transformer demonstrates remarkable adaptability and robust performance across diverse datasets.
  arXiv  Detail & Related papers  (2024-06-17T11:47:14Z)
• Unbiased Learning of Deep Generative Models with Structured Discrete Representations [7.9057320008285945]
  We propose novel algorithms for learning structured variational autoencoders (SVAEs) We are the first to demonstrate the SVAE's ability to handle multimodal uncertainty when data is missing by incorporating discrete latent variables. Our memory-efficient implicit differentiation scheme makes the SVAE tractable to learn via gradient descent, while demonstrating robustness to incomplete optimization.
  arXiv  Detail & Related papers  (2023-06-14T03:59:21Z)
• TWINS: A Fine-Tuning Framework for Improved Transferability of Adversarial Robustness and Generalization [89.54947228958494]
  This paper focuses on the fine-tuning of an adversarially pre-trained model in various classification tasks. We propose a novel statistics-based approach, Two-WIng NormliSation (TWINS) fine-tuning framework. TWINS is shown to be effective on a wide range of image classification datasets in terms of both generalization and robustness.
  arXiv  Detail & Related papers  (2023-03-20T14:12:55Z)
• Dynamically-Scaled Deep Canonical Correlation Analysis [77.34726150561087]
  Canonical Correlation Analysis (CCA) is a method for feature extraction of two views by finding maximally correlated linear projections of them. We introduce a novel dynamic scaling method for training an input-dependent canonical correlation model.
  arXiv  Detail & Related papers  (2022-03-23T12:52:49Z)
• Improving Neural Predictivity in the Visual Cortex with Gated Recurrent Connections [0.0]
  We aim to shift the focus on architectures that take into account lateral recurrent connections, a ubiquitous feature of the ventral visual stream, to devise adaptive receptive fields. In order to increase the robustness of our approach and the biological fidelity of the activations, we employ specific data augmentation techniques.
  arXiv  Detail & Related papers  (2022-03-22T17:27:22Z)
• Data-driven emergence of convolutional structure in neural networks [83.4920717252233]
  We show how fully-connected neural networks solving a discrimination task can learn a convolutional structure directly from their inputs. By carefully designing data models, we show that the emergence of this pattern is triggered by the non-Gaussian, higher-order local structure of the inputs.
  arXiv  Detail & Related papers  (2022-02-01T17:11:13Z)
• Adaptive Memory Networks with Self-supervised Learning for Unsupervised Anomaly Detection [54.76993389109327]
  Unsupervised anomaly detection aims to build models to detect unseen anomalies by only training on the normal data. We propose a novel approach called Adaptive Memory Network with Self-supervised Learning (AMSL) to address these challenges. AMSL incorporates a self-supervised learning module to learn general normal patterns and an adaptive memory fusion module to learn rich feature representations.
  arXiv  Detail & Related papers  (2022-01-03T03:40:21Z)
• Gone Fishing: Neural Active Learning with Fisher Embeddings [55.08537975896764]
  There is an increasing need for active learning algorithms that are compatible with deep neural networks. This article introduces BAIT, a practical representation of tractable, and high-performing active learning algorithm for neural networks.
  arXiv  Detail & Related papers  (2021-06-17T17:26:31Z)
• Sparse Coding Driven Deep Decision Tree Ensembles for Nuclear Segmentation in Digital Pathology Images [15.236873250912062]
  We propose an easily trained yet powerful representation learning approach with performance highly competitive to deep neural networks in a digital pathology image segmentation task. The method, called sparse coding driven deep decision tree ensembles that we abbreviate as ScD2TE, provides a new perspective on representation learning.
  arXiv  Detail & Related papers  (2020-08-13T02:59:31Z)
• Self-Supervised Learning of a Biologically-Inspired Visual Texture Model [6.931125029302013]
  We develop a model for representing visual texture in a low-dimensional feature space. Inspired by the architecture of primate visual cortex, the model uses a first stage of oriented linear filters. We show that the learned model exhibits stronger representational similarity to texture responses of neural populations recorded in primate V2 than pre-trained deep CNNs.
  arXiv  Detail & Related papers  (2020-06-30T17:12:09Z)
• MetricUNet: Synergistic Image- and Voxel-Level Learning for Precise CT Prostate Segmentation via Online Sampling [66.01558025094333]
  We propose a two-stage framework, with the first stage to quickly localize the prostate region and the second stage to precisely segment the prostate. We introduce a novel online metric learning module through voxel-wise sampling in the multi-task network. Our method can effectively learn more representative voxel-level features compared with the conventional learning methods with cross-entropy or Dice loss.
  arXiv  Detail & Related papers  (2020-05-15T10:37:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.