Sparse Autoencoder Decomposition of Clinical Sequence Model Representations: Feature Complexity, Task Specialisation, and Mortality Prediction

• URL: http://arxiv.org/abs/2605.04072v1
• Date: Mon, 13 Apr 2026 12:08:32 GMT
• Title: Sparse Autoencoder Decomposition of Clinical Sequence Model Representations: Feature Complexity, Task Specialisation, and Mortality Prediction
• Abstract summary: Sparse autoencoders (SAEs) have been applied to large language models and protein language models, but not systematically to electronic health record (EHR) foundation models.<n>We train TopK SAEs on FlatASCEND, a 14.5-million- parameter autoregressive clinical sequence model, at all 10 residual stream extraction points on INSPECT (outpatient) and MIMIC-IV (ICU)<n>SAE decomposition reveals progressive abstraction across transformer depth: layer-0 features are near-perfect token detectors (45.7% singleton), while layer-6 features span approximately 30 token types across multiple clinical categories (0.5% singleton)
• Score: 1.7865154997539017
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Sparse autoencoders (SAEs) have been applied to large language models and protein language models, but not systematically to electronic health record (EHR) foundation models. We train TopK SAEs on FlatASCEND, a 14.5-million-parameter autoregressive clinical sequence model, at all 10 residual stream extraction points on INSPECT (outpatient) and MIMIC-IV (ICU). SAE decomposition reveals progressive abstraction across transformer depth: layer-0 features are near-perfect token detectors (45.7% singleton), while layer-6 features span approximately 30 token types across multiple clinical categories (0.5% singleton). Under full-sequence simple linear probes, SAE features outperform dense representations for discrete event prediction (mortality) while dense representations outperform for continuous magnitude prediction (length of stay) - a probe-level representational phenomenon that does not extend to clinically relevant leakage-safe windows, where dense representations match or exceed SAE features across all tested settings (eICU-CRD 48-hour AUC: SAE 0.871 versus dense 0.880; base model zero-shot, SAE dictionaries trained on eICU activations; MIMIC-IV: 0.836 versus 0.914; INSPECT 1-year/3-year: 0.697 versus 0.800). A delta-mode intervention method reduces SAE perturbation noise by 86x, enabling cleaner feature-level experiments, though the resulting perturbation effects are larger than random controls in 3 of 4 conditions but not formally significant. Feature reproducibility across random seeds is 21%, and individual features should be interpreted as illustrative rather than stable.

Related papers

• Modeling Normal Is All You Need: Joint Latent Clustering for Anomaly Detection in Multimodal Cyber-Physical Systems [47.027290803102666]
  Normal behaviour is the union of many imbalanced, curved, thin-fringed operating regimes.<n>We model the normal law with a jointly learned latent representation plus explicit Gaussian-mixture mode clustering.<n>We evaluate under a deliberately fair protocol: raw point-wise metrics with no point adjustment, a trivial-detector difficulty split, prevalence-matched F1, and train-normal-only calibration.
  arXiv  Detail & Related papers  (2026-07-07T10:10:00Z)
• Cascade Classification of Dermoscopic Images of Skin Neoplasms with Controllable Sensitivity and External Clinical Validation [36.94429692322632]
  Four architectures were compared in three schemes: binary (malignant/benign), single-stage four-class (benign, MEL, SCC, BCC), and a two-stage cascade (binary triage, then three-class differentiation MEL/ SCC/BCC)<n>All models used ImageNet-pretrained weights and a single augmentation protocol on aggregated open ISIC Archive data.<n>On ISIC MILK10k, direct 11-class classification yields mean-class sensitivity 0.525.
  arXiv  Detail & Related papers  (2026-06-11T09:55:57Z)
• Perplexity Can Miss SAE Feature Damage Under Quantization [0.16921396880325779]
  Quantized models are judged acceptable when perplexity or downstream accuracy remains close to the full-precision original.<n>We test this by using a frozen sparse-autoencoder (SAE) as a fixed measurement basis.
  arXiv  Detail & Related papers  (2026-06-02T01:17:05Z)
• A Nonlinear Complexity Index for Wearable PPG Cardiovascular Stability: Multiscale Validation, Systematic Evaluation Correction, and Bayesian Parameter Optimization [1.1344265020822928]
  We introduce a Stability-Constrained Cardiovascular Stability Index (SCSI) grounded in Cardiac Stability and validate it across 176,742 segments from four heterogeneous temporal scales.<n>We identify three evaluation artifacts that inflate component from true baseline to 0.5732: segment-level cross-validation leakage, test-set leakage, and pooled overweighting that conceals per-patient failure.
  arXiv  Detail & Related papers  (2026-05-11T13:55:30Z)
• BEACON: Behavioral Entropy Aggregation for Cross-Model Hallucination Detection in Large Language Models [0.0]
  We present BEACON, a black-box hallucination detection framework that operates purely on model outputs.<n>A gradient-boosted classifier trained on 7,617 labeled examples across seven benchmarks achieves 0.8123 +/- 0.0102 AUROC.<n>An efficient 5-call variant achieves 0.7795 AUROC, enabling practical deployment across black-box LLM APIs.
  arXiv  Detail & Related papers  (2026-04-20T10:32:48Z)
• Automated Detection of Dosing Errors in Clinical Trial Narratives: A Multi-Modal Feature Engineering Approach with LightGBM [1.0152838128195467]
  We present an automated system for detecting dosing errors in unstructured clinical trial narratives.<n>Our approach combines 3,451 features spanning traditional NLP (TF-IDF, character n-grams), dense semantic embeddings, domain-specific medical patterns, and transformer-based scores.
  arXiv  Detail & Related papers  (2026-03-25T14:56:34Z)
• Cerebra: A Multidisciplinary AI Board for Multimodal Dementia Characterization and Risk Assessment [56.62016795093786]
  Cerebra is an interactive multi-agent AI team that coordinates specialized agents for EHR, clinical notes, and medical imaging analysis.<n>It supports privacy-preserving deployment by operating on structured representations and remains robust when modalities are incomplete.<n>Cerebra significantly improved expert performance, increasing accuracy by 17.5 percentage points in prospective dementia risk estimation.
  arXiv  Detail & Related papers  (2026-03-23T05:46:45Z)
• Detecting AI Hallucinations in Finance: An Information-Theoretic Method Cuts Hallucination Rate by 92% [4.693270291878929]
  Large language models (LLMs) produce fluent but unsupported answers - hallucinations.<n>We propose ECLIPSE, a framework that treats hallucination as a mismatch between a model's semantic entropy and the capacity of available evidence.
  arXiv  Detail & Related papers  (2025-12-02T05:25:48Z)
• Transparent Early ICU Mortality Prediction with Clinical Transformer and Per-Case Modality Attribution [42.85462513661566]
  We present a lightweight, transparent multimodal ensemble that fuses physiological time-series measurements with unstructured clinical notes from the first 48 hours of an ICU stay.<n>A logistic regression model combines predictions from two modality-specific models: a bidirectional LSTM for vitals and a finetuned ClinicalModernBERT transformer for notes.<n>On the MIMIC-III benchmark, our late-fusion ensemble improves discrimination over the best single model while maintaining well-calibrated predictions.
  arXiv  Detail & Related papers  (2025-11-19T20:11:49Z)
• Agentic World Modeling for 6G: Near-Real-Time Generative State-Space Reasoning [70.56067503630486]
  We argue that sixth-generation (6G) intelligence is not fluent token prediction but calibrated the capacity to imagine and choose.<n>We show that WM-MS3M cuts mean absolute error (MAE) by 1.69% versus MS3M with 32% fewer parameters and similar latency, and achieves 35-80% lower root mean squared error (RMSE) than attention/hybrid baselines with 2.3-4.1x faster inference.
  arXiv  Detail & Related papers  (2025-11-04T17:22:22Z)
• VesSAM: Efficient Multi-Prompting for Segmenting Complex Vessel [68.24765319399286]
  We present VesSAM, a powerful and efficient framework tailored for 2D vessel segmentation.<n>VesSAM integrates (1) a convolutional adapter to enhance local texture features, (2) a multi-prompt encoder that fuses anatomical prompts, and (3) a lightweight mask decoder to reduce jagged artifacts.<n>VesSAM consistently outperforms state-of-the-art PEFT-based SAM variants by over 10% Dice and 13% IoU.
  arXiv  Detail & Related papers  (2025-11-02T15:47:05Z)
• Click, Predict, Trust: Clinician-in-the-Loop AI Segmentation for Lung Cancer CT-Based Prognosis within the Knowledge-to-Action Framework [0.7340285482623061]
  Lung cancer remains the leading cause of cancer mortality, with CT imaging central to screening, prognosis, and treatment.<n>Deep learning (DL) offers automation but faces barriers to clinical adoption.<n>This study develops a clinician-in-the-loop DL pipeline to enhance, prognostic accuracy, and clinical trust.
  arXiv  Detail & Related papers  (2025-10-19T23:02:43Z)
• HANS-Net: Hyperbolic Convolution and Adaptive Temporal Attention for Accurate and Generalizable Liver and Tumor Segmentation in CT Imaging [2.7205074719266062]
  Accurate liver and tumor segmentation on abdominal CT images is critical for reliable diagnosis and treatment planning.<n>We introduce Hyperbolic-convolutions Adaptive-temporal-attention with Neural-representation and Synaptic-plasticity Network (HANS-Net)<n>HANS-Net combines hyperbolic convolutions for hierarchical geometric representation, a wavelet-inspired decomposition module for multi-scale texture learning, and an implicit neural representation branch.
  arXiv  Detail & Related papers  (2025-07-15T13:56:37Z)
• Segment-and-Classify: ROI-Guided Generalizable Contrast Phase Classification in CT Using XGBoost [7.689389068258514]
  This study utilized three public CT datasets from separate institutions.<n>The phase prediction model was trained on the WAW-TACE dataset and validated on the VinDr-Multiphase and C4KC-KiTS datasets.
  arXiv  Detail & Related papers  (2025-01-23T20:01:33Z)
• A Comprehensive Framework for Automated Segmentation of Perivascular Spaces in Brain MRI with the nnU-Net [37.179674347248266]
  Enlargement of perivascular spaces (PVS) is common in neurodegenerative disorders.<n>There is a need for reliable PVS detection methods which are currently lacking.
  arXiv  Detail & Related papers  (2024-11-29T09:19:57Z)
• CRTRE: Causal Rule Generation with Target Trial Emulation Framework [47.2836994469923]
  We introduce a novel method called causal rule generation with target trial emulation framework (CRTRE) CRTRE applies randomize trial design principles to estimate the causal effect of association rules. We then incorporate such association rules for the downstream applications such as prediction of disease onsets.
  arXiv  Detail & Related papers  (2024-11-10T02:40:06Z)
• Multilabel 12-Lead Electrocardiogram Classification Using Gradient Boosting Tree Ensemble [64.29529357862955]
  We build an algorithm using gradient boosted tree ensembles fitted on morphology and signal processing features to classify ECG diagnosis. For each lead, we derive features from heart rate variability, PQRST template shape, and the full signal waveform. We join the features of all 12 leads to fit an ensemble of gradient boosting decision trees to predict probabilities of ECG instances belonging to each class.
  arXiv  Detail & Related papers  (2020-10-21T18:11:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.