Neuron Platonic Intrinsic Representation From Dynamics Using Contrastive Learning

• URL: http://arxiv.org/abs/2502.10425v2
• Date: Tue, 18 Feb 2025 11:07:58 GMT
• Title: Neuron Platonic Intrinsic Representation From Dynamics Using Contrastive Learning
• Authors: Wei Wu, Can Liao, Zizhen Deng, Zhengrui Guo, Jinzhuo Wang,
• Abstract summary: We propose the NeurPIR (Neuron Platonic Intrinsic Representation) framework. It uses contrastive learning, with segments from the same neuron as positive pairs and those from different neurons as negative pairs. We tested our method on Izhikevich model-simulated neuronal population dynamics data.
• Score: 4.538151592901714
• License:
• Abstract: The Platonic Representation Hypothesis suggests a universal, modality-independent reality representation behind different data modalities. Inspired by this, we view each neuron as a system and detect its multi-segment activity data under various peripheral conditions. We assume there's a time-invariant representation for the same neuron, reflecting its intrinsic properties like molecular profiles, location, and morphology. The goal of obtaining these intrinsic neuronal representations has two criteria: (I) segments from the same neuron should have more similar representations than those from different neurons; (II) the representations must generalize well to out-of-domain data. To meet these, we propose the NeurPIR (Neuron Platonic Intrinsic Representation) framework. It uses contrastive learning, with segments from the same neuron as positive pairs and those from different neurons as negative pairs. In implementation, we use VICReg, which focuses on positive pairs and separates dissimilar samples via regularization. We tested our method on Izhikevich model-simulated neuronal population dynamics data. The results accurately identified neuron types based on preset hyperparameters. We also applied it to two real-world neuron dynamics datasets with neuron type annotations from spatial transcriptomics and neuron locations. Our model's learned representations accurately predicted neuron types and locations and were robust on out-of-domain data (from unseen animals). This shows the potential of our approach for understanding neuronal systems and future neuroscience research.

Related papers

• Interpreting the Second-Order Effects of Neurons in CLIP [73.54377859089801]
  We interpret the function of individual neurons in CLIP by automatically describing them using text. We present the "second-order lens", analyzing the effect flowing from a neuron through the later attention heads, directly to the output. Our results indicate that an automated interpretation of neurons can be used for model deception and for introducing new model capabilities.
  arXiv  Detail & Related papers  (2024-06-06T17:59:52Z)
• Universal Neurons in GPT2 Language Models [4.9892471449871305]
  We study the universality of individual neurons across GPT2 models trained from different initial random seeds. We find that 1-5% of neurons are universal, that is, pairs of neurons which consistently activate on the same inputs.
  arXiv  Detail & Related papers  (2024-01-22T18:11:01Z)
• Learning Time-Invariant Representations for Individual Neurons from Population Dynamics [29.936569965875375]
  We propose a self-supervised learning based method to assign time-invariant representations to individual neurons. We fit dynamical models to neuronal activity to learn a representation by considering the activity of both the individual and the neighboring population. We demonstrate our method on a public multimodal dataset of mouse cortical neuronal activity and transcriptomic labels.
  arXiv  Detail & Related papers  (2023-11-03T22:30:12Z)
• WaLiN-GUI: a graphical and auditory tool for neuron-based encoding [73.88751967207419]
  Neuromorphic computing relies on spike-based, energy-efficient communication. We develop a tool to identify suitable configurations for neuron-based encoding of sample-based data into spike trains. The WaLiN-GUI is provided open source and with documentation.
  arXiv  Detail & Related papers  (2023-10-25T20:34:08Z)
• Identifying Interpretable Visual Features in Artificial and Biological Neural Systems [3.604033202771937]
  Single neurons in neural networks are often interpretable in that they represent individual, intuitively meaningful features. Many neurons exhibit $textitmixed selectivity$, i.e., they represent multiple unrelated features. We propose an automated method for quantifying visual interpretability and an approach for finding meaningful directions in network activation space.
  arXiv  Detail & Related papers  (2023-10-17T17:41:28Z)
• Neuron to Graph: Interpreting Language Model Neurons at Scale [8.32093320910416]
  This paper introduces a novel automated approach designed to scale interpretability techniques across a vast array of neurons within Large Language Models. We propose Neuron to Graph (N2G), an innovative tool that automatically extracts a neuron's behaviour from the dataset it was trained on and translates it into an interpretable graph.
  arXiv  Detail & Related papers  (2023-05-31T14:44:33Z)
• Constraints on the design of neuromorphic circuits set by the properties of neural population codes [61.15277741147157]
  In the brain, information is encoded, transmitted and used to inform behaviour. Neuromorphic circuits need to encode information in a way compatible to that used by populations of neuron in the brain.
  arXiv  Detail & Related papers  (2022-12-08T15:16:04Z)
• Drop, Swap, and Generate: A Self-Supervised Approach for Generating Neural Activity [33.06823702945747]
  We introduce a novel unsupervised approach for learning disentangled representations of neural activity called Swap-VAE. Our approach combines a generative modeling framework with an instance-specific alignment loss. We show that it is possible to build representations that disentangle neural datasets along relevant latent dimensions linked to behavior.
  arXiv  Detail & Related papers  (2021-11-03T16:39:43Z)
• The Neural Coding Framework for Learning Generative Models [91.0357317238509]
  We propose a novel neural generative model inspired by the theory of predictive processing in the brain. In a similar way, artificial neurons in our generative model predict what neighboring neurons will do, and adjust their parameters based on how well the predictions matched reality.
  arXiv  Detail & Related papers  (2020-12-07T01:20:38Z)
• Compositional Explanations of Neurons [52.71742655312625]
  We describe a procedure for explaining neurons in deep representations by identifying compositional logical concepts. We use this procedure to answer several questions on interpretability in models for vision and natural language processing.
  arXiv  Detail & Related papers  (2020-06-24T20:37:05Z)
• Non-linear Neurons with Human-like Apical Dendrite Activations [81.18416067005538]
  We show that a standard neuron followed by our novel apical dendrite activation (ADA) can learn the XOR logical function with 100% accuracy. We conduct experiments on six benchmark data sets from computer vision, signal processing and natural language processing.
  arXiv  Detail & Related papers  (2020-02-02T21:09:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.