MedHorizon: Towards Long-context Medical Video Understanding in the Wild

• URL: http://arxiv.org/abs/2605.06537v1
• Date: Thu, 07 May 2026 16:37:10 GMT
• Title: MedHorizon: Towards Long-context Medical Video Understanding in the Wild
• Abstract summary: Real clinical review often requires full-procedure video understanding.<n>Existing benchmarks often assume this evidence has already been localized through images, short clips, or pre-segmented videos.<n>We introduce MedHorizon, an in-the-wild benchmark for long-context medical video understanding.
• Score: 78.79695798197447
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Medical multimodal large language models (MLLMs) have advanced image understanding and short-video analysis, but real clinical review often requires full-procedure video understanding. Unlike general long videos, medical procedures contain highly redundant anatomical views, while decisive evidence is temporally sparse, spatially subtle, and context dependent. Existing benchmarks often assume this evidence has already been localized through images, short clips, or pre-segmented videos, leaving the retrieval-before-reasoning problem under-tested. We introduce MedHorizon, an in-the-wild benchmark for long-context medical video understanding. MedHorizon preserves 759 hours of full-length clinical procedures and provides 1,253 evidence-grounded multiple-choice questionsthat jointly evaluate sparse evidence understanding and multi-hop clinical reasoning. Its evidence is extremely sparse, with only 0.166% evidence frames on average, requiring models to search noisy procedural streams before interpreting and aggregating findings. We evaluate representative general-domain, medical-domain, and long-video MLLMs. The best model reaches only 41.1% accuracy, showing that current systems remain far from robust full-procedure understanding. Further analysis yields four key findings: performance does not scale reliably with more frames, evidence retrieval and clinical interpretation remain primary bottlenecks; these bottlenecks are rooted in weak procedural reasoning and attention drift under redundancy, and generic sampling methods only partially balances local detail with global coverage. MedHorizon provides a rigorous testbed for MLLMs that retrieve sparse evidence and reason over complete clinical workflows.

Related papers

• MedReaMM: Evaluating Large Multimodal Models on Expert-Level Clinical Diagnostic Synthesis [61.266743965495884]
  We introduce MedReaMM, a benchmark designed to evaluate models' ability to synthesize heterogeneous clinical evidence.<n>MedReaMM comprises 625 expert-validated cases with an average of 2.79 medical images per case and a total of 1,042 standardized diagnoses annotated with ICD-11 codes.<n>We evaluate 23 Large Multimodal Models (LMMs) and find that most achieve diagnostic accuracy scores below 50%, underscoring a substantial gap in multimodal diagnostic synthesis capability.
  arXiv  Detail & Related papers  (2026-08-23T09:37:53Z)
• LoMeVQA: A Comprehensive Benchmark for Longitudinal Medical VQA [23.762697050770157]
  We propose LoMeVQA, a benchmark consisting of 206K longitudinal visual question answering (VQA) pairs for temporal medical image analysis.<n>LoMeVQA covers five tasks: progress classification, progress description, progress report generation, differential region grounding, and differential region description.<n>We introduce MedLong-8B, which achieves state-of-the-art performance across all tasks.
  arXiv  Detail & Related papers  (2026-07-30T07:45:58Z)
• Moment-Video: Diagnosing Temporal Fidelity of Video MLLMs on Momentary Visual Events [52.031070006859544]
  Video multimodal large language models (MLLMs) have made rapid progress on general and long-form video understanding, yet their ability to preserve brief answer-critical visual evidence remains underexplored.<n>We introduce Moment-Video, a benchmark for diagnosing the temporal fidelity of video MLLMs through momentary visual event understanding.
  arXiv  Detail & Related papers  (2026-06-01T17:32:20Z)
• Divide-then-Diagnose: Weaving Clinician-Inspired Contexts for Ultra-Long Capsule Endoscopy Videos [83.01543151239254]
  We define a new task, diagnosis-driven CE video summarization.<n>VideoCAP is the first CE dataset with diagnosis-driven annotations.<n>DiCE is a clinician-inspired framework that mirrors the standard CE reading workflow.
  arXiv  Detail & Related papers  (2026-04-23T16:07:51Z)
• Scaling Video Pretraining for Surgical Foundation Models [51.92777479821822]
  SurgRec is a scalable and reproducible pretraining recipe for surgical video understanding.<n>We curate a large corpus of 10,535 videos and 214.5M frames spanning endoscopy, laparoscopy, cataract, and robotic surgery.
  arXiv  Detail & Related papers  (2026-03-31T16:31:25Z)
• MedScope: Incentivizing "Think with Videos" for Clinical Reasoning via Coarse-to-Fine Tool Calling [51.31633278218137]
  MedScope is a tool-using clinical video reasoning model that performs coarse-to-fine evidence seeking over long-form procedures.<n>We build ClinVideoSuite, an evidence-centric, fine-grained clinical video suite.<n>On full and fine-grained video understanding benchmarks, MedScope achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2026-02-11T09:47:02Z)
• Clinical-Prior Guided Multi-Modal Learning with Latent Attention Pooling for Gait-Based Scoliosis Screening [8.010714901985898]
  Adolescent Idiopathic Scoliosis (AIS) is a prevalent spinal deformity whose progression can be mitigated through early detection.<n>Current screening methods are subjective, difficult to scale, and reliant on specialized clinical expertise.<n>Video-based gait analysis offers a promising alternative, but current datasets and methods frequently suffer from data leakage.<n>ScoliGait is a new benchmark dataset comprising 1,572 gait video clips for training and 300 fully independent clips for testing.
  arXiv  Detail & Related papers  (2026-02-06T14:44:22Z)
• A Specialized Large Language Model for Clinical Reasoning and Diagnosis in Rare Diseases [22.51015369266496]
  RareSeek R1 attains state of the art accuracy, robust generalization, and stability under noisy or overlapping phenotypes.<n>This work advances a narrative first, knowledge integrated reasoning paradigm that shortens the diagnostic odyssey and enables auditable, clinically translatable decision support.
  arXiv  Detail & Related papers  (2025-11-18T16:29:19Z)
• Structured Outputs Enable General-Purpose LLMs to be Medical Experts [50.02627258858336]
  Large language models (LLMs) often struggle with open-ended medical questions.<n>We propose a novel approach utilizing structured medical reasoning.<n>Our approach achieves the highest Factuality Score of 85.8, surpassing fine-tuned models.
  arXiv  Detail & Related papers  (2025-03-05T05:24:55Z)
• Unsupervised Shot Boundary Detection for Temporal Segmentation of Long Capsule Endoscopy Videos [0.0]
  Physicians use Capsule Endoscopy (CE) as a non-invasive and non-surgical procedure to examine the entire gastrointestinal (GI) tract. A single CE examination could last between 8 to 11 hours generating up to 80,000 frames which is compiled as a video.
  arXiv  Detail & Related papers  (2021-10-18T07:22:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.