Brain3D: EEG-to-3D Decoding of Visual Representations via Multimodal Reasoning

• URL: http://arxiv.org/abs/2604.08068v1
• Date: Thu, 09 Apr 2026 10:35:44 GMT
• Title: Brain3D: EEG-to-3D Decoding of Visual Representations via Multimodal Reasoning
• Abstract summary: We propose Brain3D, a multimodal architecture for EEG-to-3D reconstruction based on EEG-to-image decoding.<n>Our pipeline first produces visually grounded images from EEG signals, then employs a multimodal large language model to extract structured 3D-aware descriptions.<n>By decomposing the problem into structured stages, the proposed approach avoids direct EEG-to-3D mappings and enables scalable brain-driven 3D generation.
• Score: 14.30030169416955
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Decoding visual information from electroencephalography (EEG) has recently achieved promising results, primarily focusing on reconstructing two-dimensional (2D) images from brain activity. However, the reconstruction of three-dimensional (3D) representations remains largely unexplored. This limits the geometric understanding and reduces the applicability of neural decoding in different contexts. To address this gap, we propose Brain3D, a multimodal architecture for EEG-to-3D reconstruction based on EEG-to-image decoding. It progressively transforms neural representations into the 3D domain using geometry-aware generative reasoning. Our pipeline first produces visually grounded images from EEG signals, then employs a multimodal large language model to extract structured 3D-aware descriptions, which guide a diffusion-based generation stage whose outputs are finally converted into coherent 3D meshes via a single-image-to-3D model. By decomposing the problem into structured stages, the proposed approach avoids direct EEG-to-3D mappings and enables scalable brain-driven 3D generation. We conduct a comprehensive evaluation comparing the reconstructed 3D outputs against the original visual stimuli, assessing both semantic alignment and geometric fidelity. Experimental results demonstrate strong performance of the proposed architecture, achieving up to 85.4% 10-way Top-1 EEG decoding accuracy and 0.648 CLIPScore, supporting the feasibility of multimodal EEG-driven 3D reconstruction.

Related papers

• Dual-Stream EEG Decoding for 3D Visual Perception [1.6463973642489373]
  Bio-inspired approach implements separate decoding modules for object identity and spatial orientation.<n>We employ circular regression for angle prediction and develop EEG-conditioned multiview diffusion for 3D reconstruction.
  arXiv  Detail & Related papers  (2026-06-20T18:25:16Z)
• UniUGG: Unified 3D Understanding and Generation via Geometric-Semantic Encoding [65.60549881706959]
  We introduce UniUGG, the first unified understanding and generation framework for 3D modalities.<n>Our framework employs an LLM to comprehend and decode sentences and 3D representations.<n>We propose a spatial decoder leveraging a latent diffusion model to generate high-quality 3D representations.
  arXiv  Detail & Related papers  (2025-08-16T07:27:31Z)
• 3D-Telepathy: Reconstructing 3D Objects from EEG Signals [19.548597299697796]
  Reconstructing 3D visual stimuli from Electroencephalography (EEG) data holds significant potential for applications in Brain-Computer Interfaces (BCIs)<n>We propose an innovative EEG architecture that integrates a dual self-attention mechanism.<n>We use a hybrid training strategy to train the EEG, which includes cross-attention, contrastive learning, and self-supervised learning techniques.
  arXiv  Detail & Related papers  (2025-06-27T01:26:52Z)
• Neuro-3D: Towards 3D Visual Decoding from EEG Signals [49.502364730056044]
  We introduce a new neuroscience task: decoding 3D visual perception from EEG signals.<n>We first present EEG-3D, a dataset featuring multimodal analysis data and EEG recordings from 12 subjects viewing 72 categories of 3D objects rendered in both videos and images.<n>We propose Neuro-3D, a 3D visual decoding framework based on EEG signals.
  arXiv  Detail & Related papers  (2024-11-19T05:52:17Z)
• EEG-Driven 3D Object Reconstruction with Style Consistency and Diffusion Prior [1.7205106391379026]
  This paper proposes an EEG-based 3D object reconstruction method with style consistency and diffusion priors. Through experimental validation, we demonstrate that this method can effectively use EEG data to reconstruct 3D objects with style consistency.
  arXiv  Detail & Related papers  (2024-10-28T12:59:24Z)
• Brain3D: Generating 3D Objects from fMRI [78.46936519561298]
  We design a novel 3D object representation learning method, Brain3D, that takes as input the fMRI data of a subject.<n>We show that our model captures the distinct functionalities of each region of human vision system.<n>Preliminary evaluations indicate that Brain3D can successfully identify the disordered brain regions in simulated scenarios.
  arXiv  Detail & Related papers  (2024-05-24T06:06:11Z)
• IM-3D: Iterative Multiview Diffusion and Reconstruction for High-Quality 3D Generation [96.32684334038278]
  In this paper, we explore the design space of text-to-3D models. We significantly improve multi-view generation by considering video instead of image generators. Our new method, IM-3D, reduces the number of evaluations of the 2D generator network 10-100x.
  arXiv  Detail & Related papers  (2024-02-13T18:59:51Z)
• MinD-3D: Reconstruct High-quality 3D objects in Human Brain [50.534007259536715]
  Recon3DMind is an innovative task aimed at reconstructing 3D visuals from Functional Magnetic Resonance Imaging (fMRI) signals. We present the fMRI-Shape dataset, which includes data from 14 participants and features 360-degree videos of 3D objects. We propose MinD-3D, a novel and effective three-stage framework specifically designed to decode the brain's 3D visual information from fMRI signals.
  arXiv  Detail & Related papers  (2023-12-12T18:21:36Z)
• Improving 3D-aware Image Synthesis with A Geometry-aware Discriminator [68.0533826852601]
  3D-aware image synthesis aims at learning a generative model that can render photo-realistic 2D images while capturing decent underlying 3D shapes. Existing methods fail to obtain moderate 3D shapes. We propose a geometry-aware discriminator to improve 3D-aware GANs.
  arXiv  Detail & Related papers  (2022-09-30T17:59:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.