Learning Multimodal Volumetric Features for Large-Scale Neuron Tracing

• URL: http://arxiv.org/abs/2401.03043v1
• Date: Fri, 5 Jan 2024 19:45:12 GMT
• Title: Learning Multimodal Volumetric Features for Large-Scale Neuron Tracing
• Authors: Qihua Chen, Xuejin Chen, Chenxuan Wang, Yixiong Liu, Zhiwei Xiong, Feng Wu
• Abstract summary: We aim to reduce human workload by predicting connectivity between over-segmented neuron pieces. We first construct a dataset, named FlyTracing, that contains millions of pairwise connections of segments expanding the whole fly brain. We propose a novel connectivity-aware contrastive learning method to generate dense volumetric EM image embedding.
• Score: 72.45257414889478
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The current neuron reconstruction pipeline for electron microscopy (EM) data usually includes automatic image segmentation followed by extensive human expert proofreading. In this work, we aim to reduce human workload by predicting connectivity between over-segmented neuron pieces, taking both microscopy image and 3D morphology features into account, similar to human proofreading workflow. To this end, we first construct a dataset, named FlyTracing, that contains millions of pairwise connections of segments expanding the whole fly brain, which is three orders of magnitude larger than existing datasets for neuron segment connection. To learn sophisticated biological imaging features from the connectivity annotations, we propose a novel connectivity-aware contrastive learning method to generate dense volumetric EM image embedding. The learned embeddings can be easily incorporated with any point or voxel-based morphological representations for automatic neuron tracing. Extensive comparisons of different combination schemes of image and morphological representation in identifying split errors across the whole fly brain demonstrate the superiority of the proposed approach, especially for the locations that contain severe imaging artifacts, such as section missing and misalignment. The dataset and code are available at https://github.com/Levishery/Flywire-Neuron-Tracing.

Related papers

• NeuroFly: A framework for whole-brain single neuron reconstruction [17.93211301158225]
  We introduce NeuroFly, a validated framework for large-scale automatic single neuron reconstruction. NeuroFly breaks down the process into three distinct stages: segmentation, connection, and proofreading. Our goal is to foster collaboration among researchers to address the neuron reconstruction challenge.
  arXiv  Detail & Related papers  (2024-11-07T13:56:13Z)
• MTNeuro: A Benchmark for Evaluating Representations of Brain Structure Across Multiple Levels of Abstraction [0.0]
  In brain mapping, learning to automatically parse images to build representations of both small-scale features and global properties is a crucial and open challenge. Our benchmark (MTNeuro) is built on volumetric, micrometer-resolution X-ray microtomography images spanning a large section of mouse brain. We generated a number of different prediction challenges and evaluated several supervised and self-supervised models for brain-region prediction and pixel-level semantic segmentation of microstructures.
  arXiv  Detail & Related papers  (2023-01-01T04:54:03Z)
• Two-Stream Graph Convolutional Network for Intra-oral Scanner Image Segmentation [133.02190910009384]
  We propose a two-stream graph convolutional network (i.e., TSGCN) to handle inter-view confusion between different raw attributes. Our TSGCN significantly outperforms state-of-the-art methods in 3D tooth (surface) segmentation.
  arXiv  Detail & Related papers  (2022-04-19T10:41:09Z)
• FuNNscope: Visual microscope for interactively exploring the loss landscape of fully connected neural networks [77.34726150561087]
  We show how to explore high-dimensional landscape characteristics of neural networks. We generalize observations on small neural networks to more complex systems. An interactive dashboard opens up a number of possible application networks.
  arXiv  Detail & Related papers  (2022-04-09T16:41:53Z)
• Data-driven emergence of convolutional structure in neural networks [83.4920717252233]
  We show how fully-connected neural networks solving a discrimination task can learn a convolutional structure directly from their inputs. By carefully designing data models, we show that the emergence of this pattern is triggered by the non-Gaussian, higher-order local structure of the inputs.
  arXiv  Detail & Related papers  (2022-02-01T17:11:13Z)
• Overcoming the Domain Gap in Neural Action Representations [60.47807856873544]
  3D pose data can now be reliably extracted from multi-view video sequences without manual intervention. We propose to use it to guide the encoding of neural action representations together with a set of neural and behavioral augmentations. To reduce the domain gap, during training, we swap neural and behavioral data across animals that seem to be performing similar actions.
  arXiv  Detail & Related papers  (2021-12-02T12:45:46Z)
• Convolutional Neural Networks for cytoarchitectonic brain mapping at large scale [0.33727511459109777]
  We present a new workflow for mapping cytoarchitectonic areas in large series of cell-body stained histological sections of human postmortem brains. It is based on a Deep Convolutional Neural Network (CNN), which is trained on a pair of section images with annotations, with a large number of un-annotated sections in between. The new workflow does not require preceding 3D-reconstruction of sections, and is robust against histological artefacts.
  arXiv  Detail & Related papers  (2020-11-25T16:25:13Z)
• Deep Hypergraph U-Net for Brain Graph Embedding and Classification [0.0]
  Network neuroscience examines the brain as a system represented by a network (or connectome) We propose Hypergraph U-Net, a novel data embedding framework leveraging the hypergraph structure to learn low-dimensional embeddings of data samples. We tested our method on small-scale and large-scale heterogeneous brain connectomic datasets including morphological and functional brain networks of autistic and demented patients.
  arXiv  Detail & Related papers  (2020-08-30T08:15:18Z)
• Interpretation of 3D CNNs for Brain MRI Data Classification [56.895060189929055]
  We extend the previous findings in gender differences from diffusion-tensor imaging on T1 brain MRI scans. We provide the voxel-wise 3D CNN interpretation comparing the results of three interpretation methods.
  arXiv  Detail & Related papers  (2020-06-20T17:56:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.