BolT: Fused Window Transformers for fMRI Time Series Analysis

• URL: http://arxiv.org/abs/2205.11578v1
• Date: Mon, 23 May 2022 19:17:06 GMT
• Title: BolT: Fused Window Transformers for fMRI Time Series Analysis
• Authors: Hasan Atakan Bedel, Irmak \c{S}{\i}vg{\i}n, Onat Dalmaz, Salman Ul Hassan Dar, Tolga \c{C}ukur
• Abstract summary: We present BolT, a blood-oxygen-level-dependent transformer, for analyzing fMRI time series. To integrate information across windows, cross-window attention is computed between base tokens in each time window and fringe tokens from neighboring time windows. Experiments on public fMRI datasets clearly illustrate the superior performance of BolT against state-of-the-art methods.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Functional magnetic resonance imaging (fMRI) enables examination of inter-regional interactions in the brain via functional connectivity (FC) analyses that measure the synchrony between the temporal activations of separate regions. Given their exceptional sensitivity, deep-learning methods have received growing interest for FC analyses of high-dimensional fMRI data. In this domain, models that operate directly on raw time series as opposed to pre-computed FC features have the potential benefit of leveraging the full scale of information present in fMRI data. However, previous models are based on architectures suboptimal for temporal integration of representations across multiple time scales. Here, we present BolT, blood-oxygen-level-dependent transformer, for analyzing multi-variate fMRI time series. BolT leverages a cascade of transformer encoders equipped with a novel fused window attention mechanism. Transformer encoding is performed on temporally-overlapped time windows within the fMRI time series to capture short time-scale representations. To integrate information across windows, cross-window attention is computed between base tokens in each time window and fringe tokens from neighboring time windows. To transition from local to global representations, the extent of window overlap and thereby number of fringe tokens is progressively increased across the cascade. Finally, a novel cross-window regularization is enforced to align the high-level representations of global $CLS$ features across time windows. Comprehensive experiments on public fMRI datasets clearly illustrate the superior performance of BolT against state-of-the-art methods. Posthoc explanatory analyses to identify landmark time points and regions that contribute most significantly to model decisions corroborate prominent neuroscientific findings from recent fMRI studies.

Related papers

• Rethinking Transformer for Long Contextual Histopathology Whole Slide Image Analysis [9.090504201460817]
  Histo Whole Slide Image (WSI) analysis serves as the gold standard for clinical cancer diagnosis in the daily routines of doctors. Previous methods typically employ Multi-pathology Learning to enable slide-level prediction given only slide-level labels. To alleviate the computational complexity of long sequences in large WSIs, methods like HIPT use region-slicing, and TransMIL employs approximation of full self-attention.
  arXiv  Detail & Related papers  (2024-10-18T06:12:36Z)
• Accelerated Multi-Contrast MRI Reconstruction via Frequency and Spatial Mutual Learning [50.74383395813782]
  We propose a novel Frequency and Spatial Mutual Learning Network (FSMNet) to explore global dependencies across different modalities. The proposed FSMNet achieves state-of-the-art performance for the Multi-Contrast MR Reconstruction task with different acceleration factors.
  arXiv  Detail & Related papers  (2024-09-21T12:02:47Z)
• Generalized Dynamic Brain Functional Connectivity Based on Random Convolutions [15.620523540831021]
  We propose a generalized approach to dynamics via a multi-dimensional random convolution (RandCon) DFC method. RandCon with the smallest kernel size (3 time points) showed notable improvements in performance on simulated data. Results from real fMRI data indicated that RandCon was more sensitive to gender differences than competing methods.
  arXiv  Detail & Related papers  (2024-06-24T13:02:36Z)
• Score-CDM: Score-Weighted Convolutional Diffusion Model for Multivariate Time Series Imputation [0.035984704795350306]
  Multivariant time series (MTS) data are usually incomplete in real scenarios. We propose a Score-weighted Convolutional Diffusion Model (Score-CDM) for short, whose backbone consists of a Score-weighted Convolution Module (SCM) and an Adaptive Reception Module (ARM) We conduct extensive evaluations on three real MTS datasets of different domains, and the result verifies the effectiveness of the proposed Score-CDM.
  arXiv  Detail & Related papers  (2024-05-21T02:00:55Z)
• Fully-Connected Spatial-Temporal Graph for Multivariate Time-Series Data [50.84488941336865]
  We propose a novel method called Fully- Spatial-Temporal Graph Neural Network (FC-STGNN) For graph construction, we design a decay graph to connect sensors across all timestamps based on their temporal distances. For graph convolution, we devise FC graph convolution with a moving-pooling GNN layer to effectively capture the ST dependencies for learning effective representations.
  arXiv  Detail & Related papers  (2023-09-11T08:44:07Z)
• Diagnostic Spatio-temporal Transformer with Faithful Encoding [54.02712048973161]
  This paper addresses the task of anomaly diagnosis when the underlying data generation process has a complex-temporal (ST) dependency. We formalize the problem as supervised dependency discovery, where the ST dependency is learned as a side product of time-series classification. We show that temporal positional encoding used in existing ST transformer works has a serious limitation capturing frequencies in higher frequencies (short time scales) We also propose a new ST dependency discovery framework, which can provide readily consumable diagnostic information in both spatial and temporal directions.
  arXiv  Detail & Related papers  (2023-05-26T05:31:23Z)
• HyperTime: Implicit Neural Representation for Time Series [131.57172578210256]
  Implicit neural representations (INRs) have recently emerged as a powerful tool that provides an accurate and resolution-independent encoding of data. In this paper, we analyze the representation of time series using INRs, comparing different activation functions in terms of reconstruction accuracy and training convergence speed. We propose a hypernetwork architecture that leverages INRs to learn a compressed latent representation of an entire time series dataset.
  arXiv  Detail & Related papers  (2022-08-11T14:05:51Z)
• Contrastive predictive coding for Anomaly Detection in Multi-variate Time Series Data [6.463941665276371]
  We propose a Time-series Representational Learning through Contrastive Predictive Coding (TRL-CPC) towards anomaly detection in MVTS data. First, we jointly optimize an encoder, an auto-regressor and a non-linear transformation function to effectively learn the representations of the MVTS data sets.
  arXiv  Detail & Related papers  (2022-02-08T04:25:29Z)
• Temporal Memory Relation Network for Workflow Recognition from Surgical Video [53.20825496640025]
  We propose a novel end-to-end temporal memory relation network (TMNet) for relating long-range and multi-scale temporal patterns. We have extensively validated our approach on two benchmark surgical video datasets.
  arXiv  Detail & Related papers  (2021-03-30T13:20:26Z)
• Spectral Temporal Graph Neural Network for Multivariate Time-series Forecasting [19.50001395081601]
  StemGNN captures inter-series correlations and temporal dependencies. It can be predicted effectively by convolution and sequential learning modules. We conduct extensive experiments on ten real-world datasets to demonstrate the effectiveness of StemGNN.
  arXiv  Detail & Related papers  (2021-03-13T13:44:20Z)
• Multi-Temporal Convolutions for Human Action Recognition in Videos [83.43682368129072]
  We present a novel temporal-temporal convolution block that is capable of extracting at multiple resolutions. The proposed blocks are lightweight and can be integrated into any 3D-CNN architecture.
  arXiv  Detail & Related papers  (2020-11-08T10:40:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.