A global predicted-fMRI drive signal from TRIBE does not predict YouTube replay heatmaps

• URL: http://arxiv.org/abs/2607.01400v2
• Date: Sat, 04 Jul 2026 08:51:34 GMT
• Title: A global predicted-fMRI drive signal from TRIBE does not predict YouTube replay heatmaps
• Abstract summary: multimodal brain-encoding models predict fMRI responses to naturalistic video with high accuracy.<n>We run TRIBE on 48 YouTube videos and reduce its predicted cortical response to a per-second engagement curve.<n>Running the probe on TRIBE's input streams reveals at most a small, borderline visual-stream signal.<n>We release code, a video-ID manifest, and a heatmap-acquisition method robust to YouTube's SABR streaming.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deep multimodal brain-encoding models now predict fMRI responses to naturalistic video with high accuracy; whether their predicted neural signals also forecast behavioral engagement is unknown. We run TRIBE, the winning model of the 2025 Algonauts challenge (Llama-3.2 + V-JEPA 2 + Wav2Vec-BERT), on 48 YouTube videos and reduce its predicted cortical response to a per-second engagement curve, the global field power. Correlated against each video's "most replayed" heatmap, a proxy for re-watch, it shows no evidence of prediction: the pooled position-controlled partial correlation is +0.058 (95% CI [-0.04, 0.15]; t(47)=1.21, p=0.23), and not above simple loudness/motion baselines. The raw correlation is also near zero; the moderate values for music videos are an onset-replay artifact. The null holds across six cortical-network readouts, value/salience ROIs, and a permutation test; a supervised leave-one-video-out probe appears to reach r=0.47 but collapses to a temporal-shape artifact under a proper position control. Running the probe on TRIBE's input streams reveals at most a small, borderline visual-stream signal (matched vs. mismatched p=0.004-0.06) and none in audio, text, or the predicted cortex. The inter-subject-correlation readout, the closest prior positive result, is unavailable from the subject-averaged released model, so we fit our own per-subject encoders on the Algonauts fMRI (validated in-domain at r=0.15 and cross-domain, Friends-to-film, at r=0.10); the predicted ISC still does not track re-watch (r=-0.04, p=0.34). We bound rather than merely fail to reject the null: a Bayes factor gives moderate evidence for it (BF01=3.2), an equivalence test excludes effects above r=0.14, and the target's split-half reliability (0.82; ceiling r=0.9) rules out a noisy-label artifact. We release code, a video-ID manifest, and a heatmap-acquisition method robust to YouTube's SABR streaming.

Related papers

• Modalities Should Talk to Each Other: Dual-Stream Multimodal Learning for Long-Horizon Influenza Forecasting [1.2599533416395763]
  Publicly available surveillance datasets typically pair numeric epidemiological signals with noisy, loosely structured, only indirectly related to near-term trends, and often lagged relative to the numeric signal.<n>We propose Dual-Stream Attention (DSA), a multimodal deep learning framework that forecasts 12-week-ahead ILI activity from a 36-week multimodal history by letting the numerical and textual streams condition each other.
  arXiv  Detail & Related papers  (2026-08-24T15:20:01Z)
• TRACE-C: Rank-Calibrated Relational Anomaly Detection for Multi-Stream Operational Telemetry [0.0]
  TRACE-C is an auditable strictly-prior rank-calibrated detector for aligned multi-stream telemetry.<n>We evaluate six Great Britain grid streams with a January-April 2019 fit, July-December 2019 development evidence, and a 2020 hold-out frozen before inspection.
  arXiv  Detail & Related papers  (2026-08-21T16:01:56Z)
• Accurate Ensembles, Fragile Narratives: Multi-Scale Stacking and a Fidelity Audit of LLM-Generated Explanations for Credit Risk [0.0]
  Credit scoring increasingly relies on models whose decision logic cannot be read off their parameters.<n>A common proposal closes that gap with a language model: compute feature attributions, hand them to an LLM, and let it write the rationale.<n>We build such a system end to end and test whether the second half of the promise holds.
  arXiv  Detail & Related papers  (2026-08-08T13:22:14Z)
• How Much Do RF Drone Benchmarks Overstate? A Controlled Study and Theory of Data Leakage in UAV Signal Identification [0.0]
  Radio-frequency (RF) sensing is a central modality for counter-unmanned-aerial-system (counter-UAS) defence.<n>Reported accuracies for RF-based drone detection and identification are often very high.<n>Many are obtained using cross-validation that splits a small number of continuous recordings into short segments.<n>This can place near-duplicate slices of the same recording in both training and test partitions, creating data leakage.
  arXiv  Detail & Related papers  (2026-07-01T14:57:44Z)
• Anticipating the Optimism Gap: Predicting Distribution-Shift Degradation of RF-Impairment Detectors from In-Distribution Statistics [0.0]
  Detectors for radio-frequency impairments are usually reported with a single AUC measured on the distribution they were tuned on.<n>We ask whether this optimism can be predicted before any out-of-distribution data is seen.
  arXiv  Detail & Related papers  (2026-06-20T14:10:59Z)
• PaSBench-Video: A Streaming Video Benchmark for Proactive Safety Warning [54.70033525672316]
  Video-capable large language models (MLLMs) could serve as always-on safety monitors that issue warnings during this window.<n>We present PaSBench-Video, a 740-video benchmark with 481 risk and 259 no-risk videos across four domains.<n>We find that no model exceeds 20.0% on our strictest metric, and recall is tightly coupled with false-positive rate.
  arXiv  Detail & Related papers  (2026-06-01T16:14:01Z)
• NumLeak: Public Numeric Benchmarks as Latent Labels in Foundation Models [0.0]
  NumLeak is a measurement framework that combines API-boundary probes on production models with a white-box controlled validation on an open causal LM.<n>Top-tier frontier LLMs recall the Fama-French market excess return at 3-seed pooled Pearson r=0.97-0.99 while staying within 0.15 within-25bps on the five sibling factors.
  arXiv  Detail & Related papers  (2026-05-28T12:52:49Z)
• Amplifying, Not Learning: Fine-Tuned AI Text Detectors Amplify a Pretrained Direction [51.56484100374058]
  Text detectors amplify a pretrained typicality axis; they do not construct an AI-vs-human boundary.<n>On raw encoders before any task supervision, projecting onto centroid(AI)-centroid(HC3) achieves NYT-vs-HC3 AUROC 0.806/0.944/0.834 across three architectures.<n>On RoBERTa-base, raw projection exceeds fine-tuning; on RoBERTa-base, full fine-tuning reduces discrimination below raw on both fluent-formal populations tested.
  arXiv  Detail & Related papers  (2026-05-20T19:08:38Z)
• Don't Pause! Every prediction matters in a streaming video [55.509551643600794]
  We introduce SPOT-Bench, featuring multi-turn proactive queries that evaluate general streaming perception and assistive capabilities.<n> SPOT-Bench comes with Timeliness-F1, a metric that measures streaming predictions by their temporal precision and balanced coverage across the entire video.<n>Our benchmark reveals: (i) offline models detect events reliably but spam predictions unprompted; (ii) post-training for silence reduces spamming but induces unresponsiveness; (iii) half of the streaming video expects no response, which we term dead-time - compute spent here does not affect response latency.
  arXiv  Detail & Related papers  (2026-04-27T11:07:03Z)
• Predicting Social Media Engagement from Emotional and Temporal Features [0.0]
  We present a machine learning approach for predicting social media engagement (comments and likes) from emotional and temporal features.<n>The dataset contains 600 songs with annotations for valence, arousal, and related sentiment metrics.<n>A multi target regression model based on HistGradientBoostingRegressor is trained on log transformed engagement ratios to address skewed targets.
  arXiv  Detail & Related papers  (2025-08-29T14:14:17Z)
• Semi-Autoregressive Streaming ASR With Label Context [70.76222767090638]
  We propose a streaming "semi-autoregressive" ASR model that incorporates the labels emitted in previous blocks as additional context. Experiments show that our method outperforms the existing streaming NAR model by 19% relative on Tedlium2, 16%/8% on Librispeech-100 clean/other test sets, and 19%/8% on the Switchboard(SWB)/Callhome(CH) test sets.
  arXiv  Detail & Related papers  (2023-09-19T20:55:58Z)
• It Takes Two: Masked Appearance-Motion Modeling for Self-supervised Video Transformer Pre-training [76.69480467101143]
  Self-supervised video transformer pre-training has recently benefited from the mask-and-predict pipeline. We explicitly investigate motion cues in videos as extra prediction target and propose our Masked Appearance-Motion Modeling framework. Our method learns generalized video representations and achieves 82.3% on Kinects-400, 71.3% on Something-Something V2, 91.5% on UCF101, and 62.5% on HMDB51.
  arXiv  Detail & Related papers  (2022-10-11T08:05:18Z)
• Less is More: Sparse Sampling for Dense Reaction Predictions [60.005266111509435]
  We present our method for 2021 Evoked Expression from Videos Challenge. Our model utilizes both audio and image modalities as inputs to predict emotion changes of viewers. The proposed method has achieved pearson's correlation score of 0.04430 on the final private test set.
  arXiv  Detail & Related papers  (2021-06-03T11:33:59Z)
• Video-based Person Re-identification without Bells and Whistles [49.51670583977911]
  Video-based person re-identification (Re-ID) aims at matching the video tracklets with cropped video frames for identifying the pedestrians under different cameras. There exists severe spatial and temporal misalignment for those cropped tracklets due to the imperfect detection and tracking results generated with obsolete methods. We present a simple re-Detect and Link (DL) module which can effectively reduce those unexpected noise through applying the deep learning-based detection and tracking on the cropped tracklets.
  arXiv  Detail & Related papers  (2021-05-22T10:17:38Z)
• Probing Model Signal-Awareness via Prediction-Preserving Input Minimization [67.62847721118142]
  We evaluate models' ability to capture the correct vulnerability signals to produce their predictions. We measure the signal awareness of models using a new metric we propose- Signal-aware Recall (SAR) The results show a sharp drop in the model's Recall from the high 90s to sub-60s with the new metric.
  arXiv  Detail & Related papers  (2020-11-25T20:05:23Z)
• Deep learning for gravitational-wave data analysis: A resampling white-box approach [62.997667081978825]
  We apply Convolutional Neural Networks (CNNs) to detect gravitational wave (GW) signals of compact binary coalescences, using single-interferometer data from LIGO detectors. CNNs were quite precise to detect noise but not sensitive enough to recall GW signals, meaning that CNNs are better for noise reduction than generation of GW triggers.
  arXiv  Detail & Related papers  (2020-09-09T03:28:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.