Learning Fingerprints for Medical Time Series with Redundancy-Constrained Information Maximization

• URL: http://arxiv.org/abs/2605.00130v1
• Date: Thu, 30 Apr 2026 18:33:40 GMT
• Title: Learning Fingerprints for Medical Time Series with Redundancy-Constrained Information Maximization
• Abstract summary: We propose a novel framework that compresses a variable-length MedTS into a fixed-size set of $k$ latent Fingerprint Tokens.<n>Our architecture employs a cross-attention bottleneck to generate these tokens and is trained with a dual-objective function.<n>This approach produces a low-dimensional, interpretable, and sample-efficient representation, where each token is encouraged to capture an independent factor of variation.
• Score: 11.233142317726971
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Learning meaningful representations from medical time series (MedTS) such as ECG or EEG signals is a critical challenge. These signals are often high-dimensional, variable-length and rife with noise. Existing self-supervised approaches, such as Masked Autoencoders (MAEs) are highly effective for pre-training general-purpose encoders. However, they do not explicitly learn compact and semantically interpretable latent representations, typically relying on heuristic aggregation strategies such as global average pooling or a designated [CLS] token. We propose a novel framework that compresses a variable-length MedTS into a fixed-size set of $k$ latent Fingerprint Tokens. Our architecture employs a cross-attention bottleneck to generate these tokens and is trained with a dual-objective function. The first objective is a reconstruction loss, which ensures the tokens are \textit{sufficient statistics} for the original data. The second, a diversity penalty based on the Total Coding Rate (TCR), explicitly minimizes the redundancy between tokens, encouraging them to become statistically \textit{disentangled} representations. We present the theoretical justification for our method, framing it as a novel \textbf{Disentangled Rate-Distortion} problem. This approach produces a low-dimensional, interpretable, and sample-efficient representation, where each token is encouraged to capture an independent factor of variation, paving the way for more robust digital biomarkers.

Related papers

• ERA: Entropy-Guided Visual Token Pruning with Rectified Attention for Efficient MLLMs [77.57757084153083]
  We propose ERA, an Entropy-guided visual token pruning framework with Rectified Attention for efficient MLLMs.<n> ERA comprises three crucial components: Dual-view Entropy Pruning (DEP), Bias-aware Token Recycling (BTR), and Logit-preserving Attention Rectification (LAR)
  arXiv  Detail & Related papers  (2026-06-30T17:20:29Z)
• RAPID: Layer-Wise Redundancy-Aware Pruning and Importance-Driven Token Merging for Efficient ViT [0.014257559724536567]
  Vision Transformers (ViTs) achieve strong performance but suffer from high computational costs due to quadratic self-attention complexity.<n>We propose a depth-aware token reduction framework that adapts reduction strategies to the layer-wise characteristics of token representations.<n>Our framework provides a training-free template for optimizing vision models by aligning reduction strategies with hierarchical feature evolution.
  arXiv  Detail & Related papers  (2026-06-06T13:13:02Z)
• UniRefiner: Teaching Pre-trained ViTs to Self-Dispose Dross via Contrastive Register [30.077265994339303]
  We argue that any token failing to encode location-aligned semantics should be treated as a spurious artifact.<n>We propose UniRefiner, a universal refinement framework that teaches pre-trained Vision Transformers to self-dispose of these artifacts.<n>Our method requires only a few epochs of fine-tuning on 5k images to refine diverse ViTs, including massive models like EVA-CLIP-8B and InternViT-6B.
  arXiv  Detail & Related papers  (2026-05-19T10:00:14Z)
• RCP: Representation Consistency Pruner for Mitigating Distribution Shift in Large Vision-Language Models [24.3914653184824]
  Large Vision-Language Models (LVLMs) suffer from prohibitive inference costs due to the massive number of visual tokens processed by the language decoder.<n>Existing pruning methods often lead to significant performance degradation because the irreversible removal of visual tokens causes a distribution shift in the hidden states that deviates from the pre-trained full-token regime.<n>We propose Consistency Representation Pruner, which integrates cumulative visual token pruning with a delayed repair mechanism.
  arXiv  Detail & Related papers  (2026-04-04T13:31:45Z)
• CaTok: Taming Mean Flows for One-Dimensional Causal Image Tokenization [122.88484422855934]
  We present CaTok, a 1D causal image tokenizer with a MeanFlow decoder.<n>By selecting tokens over time intervals, CaTok learns causal 1D representations that support both fast one-step generation and high-fidelity multi-step sampling.<n> Experiments demonstrate that CaTok achieves state-of-the-art results on ImageNet reconstruction, reaching 0.75 FID, 22.53 PSNR and 0.674 SSIM.
  arXiv  Detail & Related papers  (2026-03-06T16:39:17Z)
• Learning Representations from Incomplete EHR Data with Dual-Masked Autoencoding [3.6883644193065117]
  We propose the Augmented-Intrinsic Dual-Masked Autoencoder (AID-MAE), which learns directly from incomplete time series.<n>AID-MAE processes only the unmasked subset of tokens and consistently outperforms strong baselines.
  arXiv  Detail & Related papers  (2026-02-16T19:58:44Z)
• Medical Referring Image Segmentation via Next-Token Mask Prediction [40.827152909794336]
  Medical Referring Image (MRIS) involves segmenting target regions in medical images based on natural language descriptions.<n>We propose NTP-MRISeg, a novel framework that reformulates MRIS as an autoregressive next-token prediction task over a unified multimodal sequence of tokenized image, text, and mask representations.
  arXiv  Detail & Related papers  (2025-11-07T07:29:19Z)
• Knowledge-Informed Neural Network for Complex-Valued SAR Image Recognition [51.03674130115878]
  We introduce the Knowledge-Informed Neural Network (KINN), a lightweight framework built upon a novel "compression-aggregation-compression" architecture.<n>KINN establishes a state-of-the-art in parameter-efficient recognition, offering exceptional generalization in data-scarce and out-of-distribution scenarios.
  arXiv  Detail & Related papers  (2025-10-23T07:12:26Z)
• Exploiting Discriminative Codebook Prior for Autoregressive Image Generation [54.14166700058777]
  token-based autoregressive image generation systems first tokenize images into sequences of token indices with a codebook, and then model these sequences in an autoregressive paradigm.<n>While autoregressive generative models are trained only on index values, the prior encoded in the codebook, which contains rich token similarity information, is not exploited.<n>Recent studies have attempted to incorporate this prior by performing naive k-means clustering on the tokens, helping to facilitate the training of generative models with a reduced codebook.<n>We propose the Discriminative Codebook Prior Extractor (DCPE) as an alternative to k-means
  arXiv  Detail & Related papers  (2025-08-14T15:00:00Z)
• Bridging Continuous and Discrete Tokens for Autoregressive Visual Generation [85.82112629564942]
  We propose TokenBridge, which maintains the strong representation capacity of continuous tokens while preserving the modeling simplicity of discrete tokens.<n>We introduce a dimension-wise quantization strategy that independently discretizes each feature dimension, paired with a lightweight autoregressive prediction mechanism.<n>Our approach achieves reconstruction and generation quality on par with continuous methods while using standard categorical prediction.
  arXiv  Detail & Related papers  (2025-03-20T17:59:59Z)
• MedSpaformer: a Transferable Transformer with Multi-granularity Token Sparsification for Medical Time Series Classification [25.47662257105448]
  We introduce MedSpaformer, a transformer-based framework tailored for MedTS classification.<n>It incorporates a sparse token-based dual-attention mechanism that enables global context modeling and token sparsification.<n>Our model outperforms 13 baselines across 7 medical datasets under supervised learning.
  arXiv  Detail & Related papers  (2025-03-19T13:22:42Z)
• Semantic Equitable Clustering: A Simple and Effective Strategy for Clustering Vision Tokens [57.37893387775829]
  We introduce a fast and balanced clustering method, named Semantic Equitable Clustering (SEC)<n>SEC clusters tokens based on their global semantic relevance in an efficient, straightforward manner.<n>We propose a versatile vision backbone, SECViT, to serve as a vision language connector.
  arXiv  Detail & Related papers  (2024-05-22T04:49:00Z)
• LeMeViT: Efficient Vision Transformer with Learnable Meta Tokens for Remote Sensing Image Interpretation [37.72775203647514]
  This paper proposes to use learnable meta tokens to formulate sparse tokens, which effectively learn key information and improve inference speed. By employing Dual Cross-Attention (DCA) in the early stages with dense visual tokens, we obtain the hierarchical architecture LeMeViT with various sizes. Experimental results in classification and dense prediction tasks show that LeMeViT has a significant $1.7 times$ speedup, fewer parameters, and competitive performance compared to the baseline models.
  arXiv  Detail & Related papers  (2024-05-16T03:26:06Z)
• UGMAE: A Unified Framework for Graph Masked Autoencoders [67.75493040186859]
  We propose UGMAE, a unified framework for graph masked autoencoders. We first develop an adaptive feature mask generator to account for the unique significance of nodes. We then design a ranking-based structure reconstruction objective joint with feature reconstruction to capture holistic graph information.
  arXiv  Detail & Related papers  (2024-02-12T19:39:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.