BolT: Fused Window Transformers for fMRI Time Series Analysis
- URL: http://arxiv.org/abs/2205.11578v1
- Date: Mon, 23 May 2022 19:17:06 GMT
- Title: BolT: Fused Window Transformers for fMRI Time Series Analysis
- Authors: Hasan Atakan Bedel, Irmak \c{S}{\i}vg{\i}n, Onat Dalmaz, Salman Ul
Hassan Dar, Tolga \c{C}ukur
- Abstract summary: We present BolT, a blood-oxygen-level-dependent transformer, for analyzing fMRI time series.
To integrate information across windows, cross-window attention is computed between base tokens in each time window and fringe tokens from neighboring time windows.
Experiments on public fMRI datasets clearly illustrate the superior performance of BolT against state-of-the-art methods.
- Score: 0.0
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Functional magnetic resonance imaging (fMRI) enables examination of
inter-regional interactions in the brain via functional connectivity (FC)
analyses that measure the synchrony between the temporal activations of
separate regions. Given their exceptional sensitivity, deep-learning methods
have received growing interest for FC analyses of high-dimensional fMRI data.
In this domain, models that operate directly on raw time series as opposed to
pre-computed FC features have the potential benefit of leveraging the full
scale of information present in fMRI data. However, previous models are based
on architectures suboptimal for temporal integration of representations across
multiple time scales. Here, we present BolT, blood-oxygen-level-dependent
transformer, for analyzing multi-variate fMRI time series. BolT leverages a
cascade of transformer encoders equipped with a novel fused window attention
mechanism. Transformer encoding is performed on temporally-overlapped time
windows within the fMRI time series to capture short time-scale
representations. To integrate information across windows, cross-window
attention is computed between base tokens in each time window and fringe tokens
from neighboring time windows. To transition from local to global
representations, the extent of window overlap and thereby number of fringe
tokens is progressively increased across the cascade. Finally, a novel
cross-window regularization is enforced to align the high-level representations
of global $CLS$ features across time windows. Comprehensive experiments on
public fMRI datasets clearly illustrate the superior performance of BolT
against state-of-the-art methods. Posthoc explanatory analyses to identify
landmark time points and regions that contribute most significantly to model
decisions corroborate prominent neuroscientific findings from recent fMRI
studies.
Related papers
- MMR-Mamba: Multi-Modal MRI Reconstruction with Mamba and Spatial-Frequency Information Fusion [17.084083262801737]
We propose MMR-Mamba, a novel framework that thoroughly and efficiently integrates multi-modal features for MRI reconstruction.
Specifically, we first design a Target modality-guided Cross Mamba (TCM) module in the spatial domain.
Then, we introduce a Selective Frequency Fusion (SFF) module to efficiently integrate global information in the Fourier domain.
arXiv Detail & Related papers (2024-06-27T07:30:54Z) - Score-CDM: Score-Weighted Convolutional Diffusion Model for Multivariate Time Series Imputation [0.035984704795350306]
Multivariant time series (MTS) data are usually incomplete in real scenarios.
We propose a Score-weighted Convolutional Diffusion Model (Score-CDM) for short, whose backbone consists of a Score-weighted Convolution Module (SCM) and an Adaptive Reception Module (ARM)
We conduct extensive evaluations on three real MTS datasets of different domains, and the result verifies the effectiveness of the proposed Score-CDM.
arXiv Detail & Related papers (2024-05-21T02:00:55Z) - Fully-Connected Spatial-Temporal Graph for Multivariate Time-Series Data [50.84488941336865]
We propose a novel method called Fully- Spatial-Temporal Graph Neural Network (FC-STGNN)
For graph construction, we design a decay graph to connect sensors across all timestamps based on their temporal distances.
For graph convolution, we devise FC graph convolution with a moving-pooling GNN layer to effectively capture the ST dependencies for learning effective representations.
arXiv Detail & Related papers (2023-09-11T08:44:07Z) - Coupled Attention Networks for Multivariate Time Series Anomaly
Detection [10.620044922371177]
We propose a coupled attention-based neural network framework (CAN) for anomaly detection in multivariate time series data.
To capture inter-sensor relationships and temporal dependencies, a convolutional neural network based on the global-local graph is integrated with a temporal self-attention module.
arXiv Detail & Related papers (2023-06-12T13:42:56Z) - Diagnostic Spatio-temporal Transformer with Faithful Encoding [54.02712048973161]
This paper addresses the task of anomaly diagnosis when the underlying data generation process has a complex-temporal (ST) dependency.
We formalize the problem as supervised dependency discovery, where the ST dependency is learned as a side product of time-series classification.
We show that temporal positional encoding used in existing ST transformer works has a serious limitation capturing frequencies in higher frequencies (short time scales)
We also propose a new ST dependency discovery framework, which can provide readily consumable diagnostic information in both spatial and temporal directions.
arXiv Detail & Related papers (2023-05-26T05:31:23Z) - Transform Once: Efficient Operator Learning in Frequency Domain [69.74509540521397]
We study deep neural networks designed to harness the structure in frequency domain for efficient learning of long-range correlations in space or time.
This work introduces a blueprint for frequency domain learning through a single transform: transform once (T1)
arXiv Detail & Related papers (2022-11-26T01:56:05Z) - HyperTime: Implicit Neural Representation for Time Series [131.57172578210256]
Implicit neural representations (INRs) have recently emerged as a powerful tool that provides an accurate and resolution-independent encoding of data.
In this paper, we analyze the representation of time series using INRs, comparing different activation functions in terms of reconstruction accuracy and training convergence speed.
We propose a hypernetwork architecture that leverages INRs to learn a compressed latent representation of an entire time series dataset.
arXiv Detail & Related papers (2022-08-11T14:05:51Z) - Contrastive predictive coding for Anomaly Detection in Multi-variate
Time Series Data [6.463941665276371]
We propose a Time-series Representational Learning through Contrastive Predictive Coding (TRL-CPC) towards anomaly detection in MVTS data.
First, we jointly optimize an encoder, an auto-regressor and a non-linear transformation function to effectively learn the representations of the MVTS data sets.
arXiv Detail & Related papers (2022-02-08T04:25:29Z) - Temporal Memory Relation Network for Workflow Recognition from Surgical
Video [53.20825496640025]
We propose a novel end-to-end temporal memory relation network (TMNet) for relating long-range and multi-scale temporal patterns.
We have extensively validated our approach on two benchmark surgical video datasets.
arXiv Detail & Related papers (2021-03-30T13:20:26Z) - Spectral Temporal Graph Neural Network for Multivariate Time-series
Forecasting [19.50001395081601]
StemGNN captures inter-series correlations and temporal dependencies.
It can be predicted effectively by convolution and sequential learning modules.
We conduct extensive experiments on ten real-world datasets to demonstrate the effectiveness of StemGNN.
arXiv Detail & Related papers (2021-03-13T13:44:20Z) - Multi-Temporal Convolutions for Human Action Recognition in Videos [83.43682368129072]
We present a novel temporal-temporal convolution block that is capable of extracting at multiple resolutions.
The proposed blocks are lightweight and can be integrated into any 3D-CNN architecture.
arXiv Detail & Related papers (2020-11-08T10:40:26Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.