Combining 3D Image and Tabular Data via the Dynamic Affine Feature Map Transform

• URL: http://arxiv.org/abs/2107.05990v1
• Date: Tue, 13 Jul 2021 11:18:22 GMT
• Title: Combining 3D Image and Tabular Data via the Dynamic Affine Feature Map Transform
• Authors: Sebastian P\"olsterl and Tom Nuno Wolf and Christian Wachinger
• Abstract summary: We introduce the Dynamic Affine Feature Map Transform (DAFT), a general-purpose module for CNNs that dynamically rescales and shifts the feature maps of a convolutional layer, conditional on a patient's clinical information. We show that DAFT is highly effective in combining 3D image and tabular information for diagnosis and time-to-dementia prediction, where it outperforms competing CNNs with a mean balanced accuracy of 0.622 and mean c-index of 0.748, respectively.
• Score: 3.5235974685889397
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Prior work on diagnosing Alzheimer's disease from magnetic resonance images of the brain established that convolutional neural networks (CNNs) can leverage the high-dimensional image information for classifying patients. However, little research focused on how these models can utilize the usually low-dimensional tabular information, such as patient demographics or laboratory measurements. We introduce the Dynamic Affine Feature Map Transform (DAFT), a general-purpose module for CNNs that dynamically rescales and shifts the feature maps of a convolutional layer, conditional on a patient's tabular clinical information. We show that DAFT is highly effective in combining 3D image and tabular information for diagnosis and time-to-dementia prediction, where it outperforms competing CNNs with a mean balanced accuracy of 0.622 and mean c-index of 0.748, respectively. Our extensive ablation study provides valuable insights into the architectural properties of DAFT. Our implementation is available at https://github.com/ai-med/DAFT.

Related papers

• fMRI-3D: A Comprehensive Dataset for Enhancing fMRI-based 3D Reconstruction [50.534007259536715]
  We present the fMRI-3D dataset, which includes data from 15 participants and showcases a total of 4768 3D objects. We propose MinD-3D, a novel framework designed to decode 3D visual information from fMRI signals.
  arXiv  Detail & Related papers  (2024-09-17T16:13:59Z)
• Self-Supervised Pretext Tasks for Alzheimer's Disease Classification using 3D Convolutional Neural Networks on Large-Scale Synthetic Neuroimaging Dataset [11.173478552040441]
  Alzheimer's Disease (AD) induces both localised and widespread neural degenerative changes throughout the brain. In this work, we evaluated several unsupervised methods to train a feature extractor for downstream AD vs. CN classification.
  arXiv  Detail & Related papers  (2024-06-20T11:26:32Z)
• Learning Multimodal Volumetric Features for Large-Scale Neuron Tracing [72.45257414889478]
  We aim to reduce human workload by predicting connectivity between over-segmented neuron pieces. We first construct a dataset, named FlyTracing, that contains millions of pairwise connections of segments expanding the whole fly brain. We propose a novel connectivity-aware contrastive learning method to generate dense volumetric EM image embedding.
  arXiv  Detail & Related papers  (2024-01-05T19:45:12Z)
• The effect of data augmentation and 3D-CNN depth on Alzheimer's Disease detection [51.697248252191265]
  This work summarizes and strictly observes best practices regarding data handling, experimental design, and model evaluation. We focus on Alzheimer's Disease (AD) detection, which serves as a paradigmatic example of challenging problem in healthcare. Within this framework, we train predictive 15 models, considering three different data augmentation strategies and five distinct 3D CNN architectures.
  arXiv  Detail & Related papers  (2023-09-13T10:40:41Z)
• JGAT: a joint spatio-temporal graph attention model for brain decoding [8.844033583141039]
  Joint kernel Graph Attention Network (JGAT) is a new multi-modal temporal graph attention network framework. It integrates the data from functional Magnetic Resonance Images (fMRI) and Diffusion Weighted Imaging (DWI) while preserving the dynamic information. We conduct brain-decoding tasks with our JGAT on four independent datasets.
  arXiv  Detail & Related papers  (2023-06-03T02:45:03Z)
• Decomposing 3D Neuroimaging into 2+1D Processing for Schizophrenia Recognition [25.80846093248797]
  We propose to process the 3D data by a 2+1D framework so that we can exploit the powerful deep 2D Convolutional Neural Network (CNN) networks pre-trained on the huge ImageNet dataset for 3D neuroimaging recognition. Specifically, 3D volumes of Magnetic Resonance Imaging (MRI) metrics are decomposed to 2D slices according to neighboring voxel positions. Global pooling is applied to remove redundant information as the activation patterns are sparsely distributed over feature maps. Channel-wise and slice-wise convolutions are proposed to aggregate the contextual information in the third dimension unprocessed by the 2D CNN model.
  arXiv  Detail & Related papers  (2022-11-21T15:22:59Z)
• Automated Model Design and Benchmarking of 3D Deep Learning Models for COVID-19 Detection with Chest CT Scans [72.04652116817238]
  We propose a differentiable neural architecture search (DNAS) framework to automatically search for the 3D DL models for 3D chest CT scans classification. We also exploit the Class Activation Mapping (CAM) technique on our models to provide the interpretability of the results.
  arXiv  Detail & Related papers  (2021-01-14T03:45:01Z)
• Fader Networks for domain adaptation on fMRI: ABIDE-II study [68.5481471934606]
  We use 3D convolutional autoencoders to build the domain irrelevant latent space image representation and demonstrate this method to outperform existing approaches on ABIDE data.
  arXiv  Detail & Related papers  (2020-10-14T16:50:50Z)
• Harnessing spatial homogeneity of neuroimaging data: patch individual filter layers for CNNs [0.0]
  We suggest a new CNN architecture that combines the idea of hierarchical abstraction in neural networks with a prior on the spatial homogeneity of neuroimaging data. By learning filters in individual image regions (patches) without sharing weights, PIF layers can learn abstract features faster and with fewer samples.
  arXiv  Detail & Related papers  (2020-07-23T10:11:43Z)
• 4D Spatio-Temporal Convolutional Networks for Object Position Estimation in OCT Volumes [69.62333053044712]
  3D convolutional neural networks (CNNs) have shown promising performance for pose estimation of a marker object using single OCT images. We extend 3D CNNs to 4D-temporal CNNs to evaluate the impact of additional temporal information for marker object tracking.
  arXiv  Detail & Related papers  (2020-07-02T12:02:20Z)
• Brain MRI-based 3D Convolutional Neural Networks for Classification of Schizophrenia and Controls [25.80846093248797]
  Convolutional Neural Network (CNN) has been successfully applied on classification of both natural images and medical images. We built 3D CNN models and compared their performance with a handcrafted feature-based machine learning approach. CNN models achieved higher cross-validation accuracy than handcrafted feature-based machine learning.
  arXiv  Detail & Related papers  (2020-03-14T10:05:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.