Tracking objects that change in appearance with phase synchrony

• URL: http://arxiv.org/abs/2410.02094v2
• Date: Thu, 10 Oct 2024 18:23:30 GMT
• Title: Tracking objects that change in appearance with phase synchrony
• Authors: Sabine Muzellec, Drew Linsley, Alekh K. Ashok, Ennio Mingolla, Girik Malik, Rufin VanRullen, Thomas Serre,
• Abstract summary: We show that a novel deep learning circuit can learn to control attention to features separately from their location in the world through neural synchrony. We compare object tracking in humans, the CV-RNN, and other deep neural networks (DNNs) using FeatureTracker: a large-scale challenge. Our CV-RNN behaved similarly to humans on the challenge, providing a computational proof-of-concept for the role of phase synchronization.
• Score: 14.784044408031098
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Objects we encounter often change appearance as we interact with them. Changes in illumination (shadows), object pose, or movement of nonrigid objects can drastically alter available image features. How do biological visual systems track objects as they change? It may involve specific attentional mechanisms for reasoning about the locations of objects independently of their appearances -- a capability that prominent neuroscientific theories have associated with computing through neural synchrony. We computationally test the hypothesis that the implementation of visual attention through neural synchrony underlies the ability of biological visual systems to track objects that change in appearance over time. We first introduce a novel deep learning circuit that can learn to precisely control attention to features separately from their location in the world through neural synchrony: the complex-valued recurrent neural network (CV-RNN). Next, we compare object tracking in humans, the CV-RNN, and other deep neural networks (DNNs), using FeatureTracker: a large-scale challenge that asks observers to track objects as their locations and appearances change in precisely controlled ways. While humans effortlessly solved FeatureTracker, state-of-the-art DNNs did not. In contrast, our CV-RNN behaved similarly to humans on the challenge, providing a computational proof-of-concept for the role of phase synchronization as a neural substrate for tracking appearance-morphing objects as they move about.

Related papers

• Learning Object-Centric Representation via Reverse Hierarchy Guidance [73.05170419085796]
  Object-Centric Learning (OCL) seeks to enable Neural Networks to identify individual objects in visual scenes. RHGNet introduces a top-down pathway that works in different ways in the training and inference processes. Our model achieves SOTA performance on several commonly used datasets.
  arXiv  Detail & Related papers  (2024-05-17T07:48:27Z)
• Neural feels with neural fields: Visuo-tactile perception for in-hand manipulation [57.60490773016364]
  We combine vision and touch sensing on a multi-fingered hand to estimate an object's pose and shape during in-hand manipulation. Our method, NeuralFeels, encodes object geometry by learning a neural field online and jointly tracks it by optimizing a pose graph problem. Our results demonstrate that touch, at the very least, refines and, at the very best, disambiguates visual estimates during in-hand manipulation.
  arXiv  Detail & Related papers  (2023-12-20T22:36:37Z)
• Probing neural representations of scene perception in a hippocampally dependent task using artificial neural networks [1.0312968200748116]
  Deep artificial neural networks (DNNs) trained through backpropagation provide effective models of the mammalian visual system. We describe a novel scene perception benchmark inspired by a hippocampal dependent task. Using a network architecture inspired by the connectivity between temporal lobe structures and the hippocampus, we demonstrate that DNNs trained using a triplet loss can learn this task.
  arXiv  Detail & Related papers  (2023-03-11T10:26:25Z)
• BI AVAN: Brain inspired Adversarial Visual Attention Network [67.05560966998559]
  We propose a brain-inspired adversarial visual attention network (BI-AVAN) to characterize human visual attention directly from functional brain activity. Our model imitates the biased competition process between attention-related/neglected objects to identify and locate the visual objects in a movie frame the human brain focuses on in an unsupervised manner.
  arXiv  Detail & Related papers  (2022-10-27T22:20:36Z)
• Learning What and Where -- Unsupervised Disentangling Location and Identity Tracking [0.44040106718326594]
  We introduce an unsupervisedd LOCation and Identity tracking system (Loci) Inspired by the dorsal-ventral pathways in the brain, Loci tackles the what-and-where binding problem by means of a self-supervised segregation mechanism. Loci may set the stage for deeper, explanation-oriented video processing.
  arXiv  Detail & Related papers  (2022-05-26T13:30:14Z)
• The Right Spin: Learning Object Motion from Rotation-Compensated Flow Fields [61.664963331203666]
  How humans perceive moving objects is a longstanding research question in computer vision. One approach to the problem is to teach a deep network to model all of these effects. We present a novel probabilistic model to estimate the camera's rotation given the motion field.
  arXiv  Detail & Related papers  (2022-02-28T22:05:09Z)
• The Challenge of Appearance-Free Object Tracking with Feedforward Neural Networks [12.081808043723937]
  $itPathTracker$ tests the ability of observers to learn to track objects solely by their motion. We find that standard 3D-convolutional deep network models struggle to solve this task. strategies for appearance-free object tracking from biological vision can inspire solutions.
  arXiv  Detail & Related papers  (2021-09-30T17:58:53Z)
• Object Based Attention Through Internal Gating [4.941630596191806]
  We propose an artificial neural network model of object-based attention. Our model captures the way in which attention is both top-down and recurrent. We find that our model replicates a range of findings from neuroscience.
  arXiv  Detail & Related papers  (2021-06-08T17:20:50Z)
• Tracking Without Re-recognition in Humans and Machines [12.591847867999636]
  We investigate if state-of-the-art deep neural networks for visual tracking are capable of the same. We introduce PathTracker, a synthetic visual challenge that asks human observers and machines to track a target object. We model circuit mechanisms in biological brains that are implicated in tracking objects based on motion cues.
  arXiv  Detail & Related papers  (2021-05-27T17:56:37Z)
• Continuity-Discrimination Convolutional Neural Network for Visual Object Tracking [150.51667609413312]
  This paper proposes a novel model, named Continuity-Discrimination Convolutional Neural Network (CD-CNN) for visual object tracking. To address this problem, CD-CNN models temporal appearance continuity based on the idea of temporal slowness. In order to alleviate inaccurate target localization and drifting, we propose a novel notion, object-centroid.
  arXiv  Detail & Related papers  (2021-04-18T06:35:03Z)
• Neuroevolution of a Recurrent Neural Network for Spatial and Working Memory in a Simulated Robotic Environment [57.91534223695695]
  We evolved weights in a biologically plausible recurrent neural network (RNN) using an evolutionary algorithm to replicate the behavior and neural activity observed in rats. Our method demonstrates how the dynamic activity in evolved RNNs can capture interesting and complex cognitive behavior.
  arXiv  Detail & Related papers  (2021-02-25T02:13:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.