Computational imaging with the human brain

• URL: http://arxiv.org/abs/2210.03400v1
• Date: Fri, 7 Oct 2022 08:40:18 GMT
• Title: Computational imaging with the human brain
• Authors: Gao Wang, Daniele Faccio
• Abstract summary: Brain-computer interfaces (BCIs) are enabling a range of new possibilities and routes for augmenting human capability. We demonstrate ghost imaging of a hidden scene using the human visual system that is combined with an adaptive computational imaging scheme. This brain-computer connectivity demonstrates a form of augmented human computation that could in the future extend the sensing range of human vision.
• Score: 1.614301262383079
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Brain-computer interfaces (BCIs) are enabling a range of new possibilities and routes for augmenting human capability. Here, we propose BCIs as a route towards forms of computation, i.e. computational imaging, that blend the brain with external silicon processing. We demonstrate ghost imaging of a hidden scene using the human visual system that is combined with an adaptive computational imaging scheme. This is achieved through a projection pattern `carving' technique that relies on real-time feedback from the brain to modify patterns at the light projector, thus enabling more efficient and higher resolution imaging. This brain-computer connectivity demonstrates a form of augmented human computation that could in the future extend the sensing range of human vision and provide new approaches to the study of the neurophysics of human perception. As an example, we illustrate a simple experiment whereby image reconstruction quality is affected by simultaneous conscious processing and readout of the perceived light intensities.

Related papers

• Brain3D: Generating 3D Objects from fMRI [76.41771117405973]
  We design a novel 3D object representation learning method, Brain3D, that takes as input the fMRI data of a subject. We show that our model captures the distinct functionalities of each region of human vision system. Preliminary evaluations indicate that Brain3D can successfully identify the disordered brain regions in simulated scenarios.
  arXiv  Detail & Related papers  (2024-05-24T06:06:11Z)
• Hebbian Learning based Orthogonal Projection for Continual Learning of Spiking Neural Networks [74.3099028063756]
  We develop a new method with neuronal operations based on lateral connections and Hebbian learning. We show that Hebbian and anti-Hebbian learning on recurrent lateral connections can effectively extract the principal subspace of neural activities. Our method consistently solves for spiking neural networks with nearly zero forgetting.
  arXiv  Detail & Related papers  (2024-02-19T09:29:37Z)
• Achieving More Human Brain-Like Vision via Human EEG Representational Alignment [1.811217832697894]
  We present 'Re(presentational)Al(ignment)net', a vision model aligned with human brain activity based on non-invasive EEG. Our innovative image-to-brain multi-layer encoding framework advances human neural alignment by optimizing multiple model layers. Our findings suggest that ReAlnet represents a breakthrough in bridging the gap between artificial and human vision, and paving the way for more brain-like artificial intelligence systems.
  arXiv  Detail & Related papers  (2024-01-30T18:18:41Z)
• Learning Multimodal Volumetric Features for Large-Scale Neuron Tracing [72.45257414889478]
  We aim to reduce human workload by predicting connectivity between over-segmented neuron pieces. We first construct a dataset, named FlyTracing, that contains millions of pairwise connections of segments expanding the whole fly brain. We propose a novel connectivity-aware contrastive learning method to generate dense volumetric EM image embedding.
  arXiv  Detail & Related papers  (2024-01-05T19:45:12Z)
• Unidirectional brain-computer interface: Artificial neural network encoding natural images to fMRI response in the visual cortex [12.1427193917406]
  We propose an artificial neural network dubbed VISION to mimic the human brain and show how it can foster neuroscientific inquiries. VISION successfully predicts human hemodynamic responses as fMRI voxel values to visual inputs with an accuracy exceeding state-of-the-art performance by 45%.
  arXiv  Detail & Related papers  (2023-09-26T15:38:26Z)
• Seeing through the Brain: Image Reconstruction of Visual Perception from Human Brain Signals [27.92796103924193]
  We propose a comprehensive pipeline, named NeuroImagen, for reconstructing visual stimuli images from EEG signals. We incorporate a novel multi-level perceptual information decoding to draw multi-grained outputs from the given EEG data.
  arXiv  Detail & Related papers  (2023-07-27T12:54:16Z)
• Brain Captioning: Decoding human brain activity into images and text [1.5486926490986461]
  We present an innovative method for decoding brain activity into meaningful images and captions. Our approach takes advantage of cutting-edge image captioning models and incorporates a unique image reconstruction pipeline. We evaluate our methods using quantitative metrics for both generated captions and images.
  arXiv  Detail & Related papers  (2023-05-19T09:57:19Z)
• Controllable Mind Visual Diffusion Model [58.83896307930354]
  Brain signal visualization has emerged as an active research area, serving as a critical interface between the human visual system and computer vision models. We propose a novel approach, referred to as Controllable Mind Visual Model Diffusion (CMVDM) CMVDM extracts semantic and silhouette information from fMRI data using attribute alignment and assistant networks. We then leverage a control model to fully exploit the extracted information for image synthesis, resulting in generated images that closely resemble the visual stimuli in terms of semantics and silhouette.
  arXiv  Detail & Related papers  (2023-05-17T11:36:40Z)
• BrainCLIP: Bridging Brain and Visual-Linguistic Representation Via CLIP for Generic Natural Visual Stimulus Decoding [51.911473457195555]
  BrainCLIP is a task-agnostic fMRI-based brain decoding model. It bridges the modality gap between brain activity, image, and text. BrainCLIP can reconstruct visual stimuli with high semantic fidelity.
  arXiv  Detail & Related papers  (2023-02-25T03:28:54Z)
• Adapting Brain-Like Neural Networks for Modeling Cortical Visual Prostheses [68.96380145211093]
  Cortical prostheses are devices implanted in the visual cortex that attempt to restore lost vision by electrically stimulating neurons. Currently, the vision provided by these devices is limited, and accurately predicting the visual percepts resulting from stimulation is an open challenge. We propose to address this challenge by utilizing 'brain-like' convolutional neural networks (CNNs), which have emerged as promising models of the visual system.
  arXiv  Detail & Related papers  (2022-09-27T17:33:19Z)
• A neuromorphic approach to image processing and machine vision [0.9137554315375922]
  We explore the implementation of visual tasks such as image segmentation, visual attention and object recognition. We have emphasized on the employment of non-volatile memory devices such as memristors to realize artificial visual systems.
  arXiv  Detail & Related papers  (2022-08-07T05:01:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.