From Sports to Safety: Benchmarking Proactive Risk Inference in MLLMs

• URL: http://arxiv.org/abs/2608.05560v1
• Date: Thu, 06 Aug 2026 03:25:39 GMT
• Title: From Sports to Safety: Benchmarking Proactive Risk Inference in MLLMs
• Abstract summary: SPRINT (Sports Proactive Risk INference Testbed) is a benchmark of 2,888 real-world sports videos (2,440 accident, 448 safe controls) spanning 14 sports and 3 environmental settings.<n> Accident videos feature fine-grained annotations of early hazard cues, accident timing, and hierarchical causes; safe videos are manually verified as accident-free and serve to diagnose prompt-induced false alarms.
• Score: 11.582231928519754
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Timely anticipation of physical hazards is essential for real-world safety, yet existing MLLM evaluations focus on harmful content or general risks, leaving proactive physical hazard prediction underexplored. Sports provide a well-suited testbed: accident causes span diverse injury dimensions and pre-accident spatiotemporal cues draw on reasoning capabilities shared with broader safety domains such as autonomous driving and fall detection. We introduce SPRINT (Sports Proactive Risk INference Testbed), a benchmark of 2,888 real-world sports videos (2,440 accident, 448 safe controls) spanning 14 sports and 3 environmental settings. Accident videos feature fine-grained annotations of early hazard cues, accident timing, and hierarchical causes; safe videos are manually verified as accident-free and serve to diagnose prompt-induced false alarms. Evaluating state-of-the-art MLLMs under diverse prompts and temporal windows reveals a sharp gap between hazard sensitivity and understanding: the best model exceeds 95% in signaling hazards yet falls below 50% in identifying their causes. Diagnostic experiments further show that explicit danger queries trigger severe false alarms even on hazard-free videos. These findings indicate that current MLLMs exhibit only superficial proactive safety, lacking stable, cause-grounded early warning, and underscore the need for reliable proactive safety in dynamic physical environments. Data and code will be open-sourced upon acceptance.

Related papers

• Hazard or Anomaly? Evaluating VLMs for Understanding Dangers and Discrepancies [3.227001704699474]
  Vision-Language Models (VLMs) can interpret complex scenes and communicate safety-relevant information.<n>Current evaluations often frame danger recognition as a binary decision (Safe/Unsafe)<n>We introduce an explicit distinction between hazard and anomaly, and by separately recognizing hazardous and anomalous states.
  arXiv  Detail & Related papers  (2026-07-18T02:24:06Z)
• EgoSafetyBench: A Diagnostic Egocentric Video Benchmark for Evaluating Embodied VLMs as Runtime Safety Guards [4.6400797556929545]
  Vision-language models (VLMs) are proposed as runtime safety guards for embodied agents in homes and factories.<n>We introduce EgoSafetyBench, an egocentric video benchmark of 1,200 robot-view scenarios annotated at half-second granularity.<n>We find that while guards reliably recognize videos containing hazards, they often miss specific hazardous moments.
  arXiv  Detail & Related papers  (2026-06-30T21:50:49Z)
• PaSBench-Video: A Streaming Video Benchmark for Proactive Safety Warning [54.70033525672316]
  Video-capable large language models (MLLMs) could serve as always-on safety monitors that issue warnings during this window.<n>We present PaSBench-Video, a 740-video benchmark with 481 risk and 259 no-risk videos across four domains.<n>We find that no model exceeds 20.0% on our strictest metric, and recall is tightly coupled with false-positive rate.
  arXiv  Detail & Related papers  (2026-06-01T16:14:01Z)
• SafetyALFRED: Evaluating Safety-Conscious Planning of Multimodal Large Language Models [46.968844120076916]
  We introduce SafetyALFRED, built upon the embodied agent benchmark ALFRED, augmented with six categories of real-world kitchen hazards.<n>We evaluate eleven state-of-the-art models on not only hazard recognition, but also active risk mitigation through embodied planning.
  arXiv  Detail & Related papers  (2026-04-21T16:27:20Z)
• The Missing Half: Unveiling Training-time Implicit Safety Risks Beyond Deployment [148.80266237240713]
  implicit training-time safety risks are driven by a model's internal incentives and contextual background information.<n>We present the first systematic study of this problem, introducing a taxonomy with five risk levels, ten fine-grained risk categories, and three incentive types.<n>Our results identify an overlooked yet urgent safety challenge in training.
  arXiv  Detail & Related papers  (2026-02-04T04:23:58Z)
• When Helpers Become Hazards: A Benchmark for Analyzing Multimodal LLM-Powered Safety in Daily Life [36.244977974241245]
  We investigate and evaluate the safety impact of Multimodal Large Language Models (MLLMs) on human behavior in daily life.<n>We introduce SaLAD, a multimodal safety benchmark which contains 2,013 real-world image-text samples.<n>Results on 18 MLLMs demonstrate that the top-performing models achieve a safe response rate of only 57.2% on unsafe queries.
  arXiv  Detail & Related papers  (2026-01-07T15:59:07Z)
• RiskCueBench: Benchmarking Anticipatory Reasoning from Early Risk Cues in Video-Language Models [4.858738694604317]
  We introduce a new video understanding benchmark RiskCueBench in which videos are carefully annotated to identify a risk signal clip.<n> Experimental results reveal a significant gap in current systems ability to interpret evolving situations and anticipate future risky events from early visual signals.
  arXiv  Detail & Related papers  (2026-01-06T19:14:49Z)
• RoboSafe: Safeguarding Embodied Agents via Executable Safety Logic [56.38397499463889]
  Embodied agents powered by vision-language models (VLMs) are increasingly capable of executing complex real-world tasks.<n>However, they remain vulnerable to hazardous instructions that may trigger unsafe behaviors.<n>We propose RoboSafe, a runtime safeguard for embodied agents through executable predicate-based safety logic.
  arXiv  Detail & Related papers  (2025-12-24T15:01:26Z)
• Video-SafetyBench: A Benchmark for Safety Evaluation of Video LVLMs [45.265397990158846]
  Video-SafetyBench is the first benchmark designed to evaluate the safety of LVLMs under video-text attacks.<n>It comprises 2,264 video-text pairs spanning 48 fine-grained unsafe categories.<n>To generate semantically accurate videos for safety evaluation, we design a controllable pipeline that decomposes video semantics into subject images and motion text.
  arXiv  Detail & Related papers  (2025-05-17T05:06:38Z)
• SafeMLRM: Demystifying Safety in Multi-modal Large Reasoning Models [50.34706204154244]
  Acquiring reasoning capabilities catastrophically degrades inherited safety alignment.<n>Certain scenarios suffer 25 times higher attack rates.<n>Despite tight reasoning-answer safety coupling, MLRMs demonstrate nascent self-correction.
  arXiv  Detail & Related papers  (2025-04-09T06:53:23Z)
• LLMs know their vulnerabilities: Uncover Safety Gaps through Natural Distribution Shifts [88.96201324719205]
  Safety concerns in large language models (LLMs) have gained significant attention due to their exposure to potentially harmful data during pre-training.<n>We identify a new safety vulnerability in LLMs, where seemingly benign prompts, semantically related to harmful content, can bypass safety mechanisms.<n>We introduce a novel attack method, textitActorBreaker, which identifies actors related to toxic prompts within pre-training distribution.
  arXiv  Detail & Related papers  (2024-10-14T16:41:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.