Biologically-Motivated Learning Model for Instructed Visual Processing

• URL: http://arxiv.org/abs/2306.02415v3
• Date: Sun, 16 Jun 2024 15:24:53 GMT
• Title: Biologically-Motivated Learning Model for Instructed Visual Processing
• Authors: Roy Abel, Shimon Ullman,
• Abstract summary: Current models of biologically plausible learning often use a cortical-like combination of bottom-up (BU) and top-down (TD) processing. In the visual cortex, the TD pathway plays a second major role of visual attention, by guiding the visual process to locations and tasks of interest. We introduce a model that uses a cortical-like combination of BU and TD processing that naturally integrates the two major functions of the TD stream.
• Score: 3.105144691395886
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As part of understanding how the brain learns, ongoing work seeks to combine biological knowledge and current artificial intelligence (AI) modeling in an attempt to find an efficient biologically plausible learning scheme. Current models of biologically plausible learning often use a cortical-like combination of bottom-up (BU) and top-down (TD) processing, where the TD part carries feedback signals used for learning. However, in the visual cortex, the TD pathway plays a second major role of visual attention, by guiding the visual process to locations and tasks of interest. A biological model should therefore combine the two tasks, and learn to guide the visual process. We introduce a model that uses a cortical-like combination of BU and TD processing that naturally integrates the two major functions of the TD stream. The integrated model is obtained by an appropriate connectivity pattern between the BU and TD streams, a novel processing cycle that uses the TD part twice, and the use of 'Counter-Hebb' learning that operates across the streams. We show that the 'Counter-Hebb' mechanism can provide an exact backpropagation synaptic modification. We further demonstrate the model's ability to guide the visual stream to perform a task of interest, achieving competitive performance compared with AI models on standard multi-task learning benchmarks. The successful combination of learning and visual guidance could provide a new view on combining BU and TD processing in human vision, and suggests possible directions for both biologically plausible models and artificial instructed models, such as vision-language models (VLMs).

Related papers

• BIMM: Brain Inspired Masked Modeling for Video Representation Learning [47.56270575865621]
  We propose the Brain Inspired Masked Modeling (BIMM) framework, aiming to learn comprehensive representations from videos. Specifically, our approach consists of ventral and dorsal branches, which learn image and video representations, respectively. To achieve the goals of different visual cortices in the brain, we segment the encoder of each branch into three intermediate blocks and reconstruct progressive prediction targets with light weight decoders.
  arXiv  Detail & Related papers  (2024-05-21T13:09:04Z)
• Harnessing Diffusion Models for Visual Perception with Meta Prompts [68.78938846041767]
  We propose a simple yet effective scheme to harness a diffusion model for visual perception tasks. We introduce learnable embeddings (meta prompts) to the pre-trained diffusion models to extract proper features for perception. Our approach achieves new performance records in depth estimation tasks on NYU depth V2 and KITTI, and in semantic segmentation task on CityScapes.
  arXiv  Detail & Related papers  (2023-12-22T14:40:55Z)
• Distilling Knowledge from CNN-Transformer Models for Enhanced Human Action Recognition [1.8722948221596285]
  The research aims to enhance the performance and efficiency of smaller student models by transferring knowledge from larger teacher models. The proposed method employs a Transformer vision network as the student model, while a convolutional network serves as the teacher model. The Vision Transformer (ViT) architecture is introduced as a robust framework for capturing global dependencies in images.
  arXiv  Detail & Related papers  (2023-11-02T14:57:58Z)
• Visual Affordance Prediction for Guiding Robot Exploration [56.17795036091848]
  We develop an approach for learning visual affordances for guiding robot exploration. We use a Transformer-based model to learn a conditional distribution in the latent embedding space of a VQ-VAE. We show how the trained affordance model can be used for guiding exploration by acting as a goal-sampling distribution, during visual goal-conditioned policy learning in robotic manipulation.
  arXiv  Detail & Related papers  (2023-05-28T17:53:09Z)
• Modelling Human Visual Motion Processing with Trainable Motion Energy Sensing and a Self-attention Network [1.9458156037869137]
  We propose an image-computable model of human motion perception by bridging the gap between biological and computer vision models. This model architecture aims to capture the computations in V1-MT, the core structure for motion perception in the biological visual system. In silico neurophysiology reveals that our model's unit responses are similar to mammalian neural recordings regarding motion pooling and speed tuning.
  arXiv  Detail & Related papers  (2023-05-16T04:16:07Z)
• Multi-Mode Online Knowledge Distillation for Self-Supervised Visual Representation Learning [13.057037169495594]
  We propose a Multi-mode Online Knowledge Distillation method (MOKD) to boost self-supervised visual representation learning. In MOKD, two different models learn collaboratively in a self-supervised manner. In addition, MOKD also outperforms existing SSL-KD methods for both the student and teacher models.
  arXiv  Detail & Related papers  (2023-04-13T12:55:53Z)
• Masked World Models for Visual Control [90.13638482124567]
  We introduce a visual model-based RL framework that decouples visual representation learning and dynamics learning. We demonstrate that our approach achieves state-of-the-art performance on a variety of visual robotic tasks.
  arXiv  Detail & Related papers  (2022-06-28T18:42:27Z)
• UViM: A Unified Modeling Approach for Vision with Learned Guiding Codes [91.24112204588353]
  We introduce UViM, a unified approach capable of modeling a wide range of computer vision tasks. In contrast to previous models, UViM has the same functional form for all tasks. We demonstrate the effectiveness of UViM on three diverse and challenging vision tasks.
  arXiv  Detail & Related papers  (2022-05-20T17:47:59Z)
• Learning Task Informed Abstractions [10.920599910769276]
  We propose learning Task Informed Abstractions (TIA) that explicitly separates reward-correlated visual features from distractors. TIA leads to significant performance gains over state-of-the-art methods on many visual control tasks.
  arXiv  Detail & Related papers  (2021-06-29T17:56:11Z)
• Relational Graph Learning on Visual and Kinematics Embeddings for Accurate Gesture Recognition in Robotic Surgery [84.73764603474413]
  We propose a novel online approach of multi-modal graph network (i.e., MRG-Net) to dynamically integrate visual and kinematics information. The effectiveness of our method is demonstrated with state-of-the-art results on the public JIGSAWS dataset.
  arXiv  Detail & Related papers  (2020-11-03T11:00:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.