Using Deep Learning for Visual Decoding and Reconstruction from Brain Activity: A Review

• URL: http://arxiv.org/abs/2108.04169v1
• Date: Mon, 9 Aug 2021 16:54:35 GMT
• Title: Using Deep Learning for Visual Decoding and Reconstruction from Brain Activity: A Review
• Authors: Madison Van Horn
• Abstract summary: I will show that these structures can struggle with adaptability to various input stimuli due to complicated objects in images. This paper will conclude the use of deep learning within visual decoding and reconstruction is highly optimal when using variations of deep neural networks.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This literature review will discuss the use of deep learning methods for image reconstruction using fMRI data. More specifically, the quality of image reconstruction will be determined by the choice in decoding and reconstruction architectures. I will show that these structures can struggle with adaptability to various input stimuli due to complicated objects in images. Also, the significance of feature representation will be evaluated. This paper will conclude the use of deep learning within visual decoding and reconstruction is highly optimal when using variations of deep neural networks and will provide details of potential future work.

Related papers

• MindFormer: A Transformer Architecture for Multi-Subject Brain Decoding via fMRI [50.55024115943266]
  We introduce a new Transformer architecture called MindFormer to generate fMRI-conditioned feature vectors. MindFormer incorporates two key innovations: 1) a novel training strategy based on the IP-Adapter to extract semantically meaningful features from fMRI signals, and 2) a subject specific token and linear layer that effectively capture individual differences in fMRI signals.
  arXiv  Detail & Related papers  (2024-05-28T00:36:25Z)
• Learning Object-Centric Representation via Reverse Hierarchy Guidance [73.05170419085796]
  Object-Centric Learning (OCL) seeks to enable Neural Networks to identify individual objects in visual scenes. RHGNet introduces a top-down pathway that works in different ways in the training and inference processes. Our model achieves SOTA performance on several commonly used datasets.
  arXiv  Detail & Related papers  (2024-05-17T07:48:27Z)
• Facial Image Reconstruction from Functional Magnetic Resonance Imaging via GAN Inversion with Improved Attribute Consistency [5.705640492618758]
  We propose a new framework to reconstruct facial images from fMRI data. The proposed framework accomplishes two goals: (1) reconstructing clear facial images from fMRI data and (2) maintaining the consistency of semantic characteristics.
  arXiv  Detail & Related papers  (2022-07-03T11:18:35Z)
• Multiscale Voxel Based Decoding For Enhanced Natural Image Reconstruction From Brain Activity [0.22940141855172028]
  We present a novel approach for enhanced image reconstruction, in which existing methods for object decoding and image reconstruction are merged together. This is achieved by conditioning the reconstructed image to its decoded image category using a class-conditional generative adversarial network and neural style transfer. The results indicate that our approach improves the semantic similarity of the reconstructed images and can be used as a general framework for enhanced image reconstruction.
  arXiv  Detail & Related papers  (2022-05-27T18:09:07Z)
• Is Deep Image Prior in Need of a Good Education? [57.3399060347311]
  Deep image prior was introduced as an effective prior for image reconstruction. Despite its impressive reconstructive properties, the approach is slow when compared to learned or traditional reconstruction techniques. We develop a two-stage learning paradigm to address the computational challenge.
  arXiv  Detail & Related papers  (2021-11-23T15:08:26Z)
• Natural Image Reconstruction from fMRI using Deep Learning: A Survey [5.821090056678976]
  We survey the most recent deep learning methods for natural image reconstruction from fMRI. We examine these methods in terms of architectural design, benchmark datasets, and evaluation metrics. We discuss the strengths and limitations of existing studies and present potential future directions.
  arXiv  Detail & Related papers  (2021-10-18T04:05:29Z)
• Deep Unrolled Recovery in Sparse Biological Imaging [62.997667081978825]
  Deep algorithm unrolling is a model-based approach to develop deep architectures that combine the interpretability of iterative algorithms with the performance gains of supervised deep learning. This framework is well-suited to applications in biological imaging, where physics-based models exist to describe the measurement process and the information to be recovered is often highly structured.
  arXiv  Detail & Related papers  (2021-09-28T20:22:44Z)
• Over-and-Under Complete Convolutional RNN for MRI Reconstruction [57.95363471940937]
  Recent deep learning-based methods for MR image reconstruction usually leverage a generic auto-encoder architecture. We propose an Over-and-Under Complete Convolu?tional Recurrent Neural Network (OUCR), which consists of an overcomplete and an undercomplete Convolutional Recurrent Neural Network(CRNN) The proposed method achieves significant improvements over the compressed sensing and popular deep learning-based methods with less number of trainable parameters.
  arXiv  Detail & Related papers  (2021-06-16T15:56:34Z)
• Deep Learning in Photoacoustic Tomography: Current approaches and future directions [2.631277214890658]
  Photoacoustic tomography can provide high resolution 3D soft tissue images based on the optical absorption. The need for rapid image formation and the practical restrictions on data acquisition are presenting new image reconstruction challenges. Deep Learning, or deep neural networks, to this problem has received a great deal of attention.
  arXiv  Detail & Related papers  (2020-09-16T11:33:29Z)
• NAS-DIP: Learning Deep Image Prior with Neural Architecture Search [65.79109790446257]
  Recent work has shown that the structure of deep convolutional neural networks can be used as a structured image prior. We propose to search for neural architectures that capture stronger image priors. We search for an improved network by leveraging an existing neural architecture search algorithm.
  arXiv  Detail & Related papers  (2020-08-26T17:59:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.