From Skeletons to Semantics: Design and Deployment of a Hybrid Edge-Based Action Detection System for Public Safety

• URL: http://arxiv.org/abs/2603.29777v1
• Date: Tue, 31 Mar 2026 14:16:16 GMT
• Title: From Skeletons to Semantics: Design and Deployment of a Hybrid Edge-Based Action Detection System for Public Safety
• Abstract summary: This paper presents the design and demonstrator-based deployment of a hybrid edge-based action detection system.<n>It combines skeleton-based motion analysis with vision-language models for semantic scene interpretation.<n>The presented system provides a practical foundation for privacy-aware, real-time video analysis in public safety applications.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Public spaces such as transport hubs, city centres, and event venues require timely and reliable detection of potentially violent behaviour to support public safety. While automated video analysis has made significant progress, practical deployment remains constrained by latency, privacy, and resource limitations, particularly under edge-computing conditions. This paper presents the design and demonstrator-based deployment of a hybrid edge-based action detection system that combines skeleton-based motion analysis with vision-language models for semantic scene interpretation. Skeleton-based processing enables continuous, privacy-aware monitoring with low computational overhead, while vision-language models provide contextual understanding and zero-shot reasoning capabilities for complex and previously unseen situations. Rather than proposing new recognition models, the contribution focuses on a system-level comparison of both paradigms under realistic edge constraints. The system is implemented on a GPU-enabled edge device and evaluated with respect to latency, resource usage, and operational trade-offs using a demonstrator-based setup. The results highlight the complementary strengths and limitations of motioncentric and semantic approaches and motivate a hybrid architecture that selectively augments fast skeletonbased detection with higher-level semantic reasoning. The presented system provides a practical foundation for privacy-aware, real-time video analysis in public safety applications.

Related papers

• Towards Generalizable Visually Grounded Exploration of Household Devices [74.66117923766632]
  Key bottleneck of realizing general autonomous embodied agents lies in Generalizable Visually Grounded Exploration.<n>We introduce VGEBench, a benchmark designed to evaluate the generalizable visually grounded exploration capabilities of Vision-Language Models.<n>Unlike static datasets, we construct a Logic-Driven State Machine framework. This framework simulates multi-turn interaction loops, compelling agents to achieve goals by active visual perception and feedback-driven correction.
  arXiv  Detail & Related papers  (2026-09-01T07:42:45Z)
• Partial Skeleton Visibility for Action Recognition: A Constrained Field-of-View Approach [60.596944437968524]
  PartialVisGraph is a novel framework for robust skeleton-based action recognition under constrained field-of-view.<n>It consistently achieves state-of-the-art accuracy under partial visibility, with gains of up to 68.8% on subsets with severe FoV restrictions.<n>Our approach offers a principled and practical pathway toward deployable skeleton-based action understanding in unconstrained environments.
  arXiv  Detail & Related papers  (2026-07-01T10:03:11Z)
• Learning to Watch: Active Video Anomaly Understanding via Interleaved Policy Optimization [56.150184676745205]
  Video anomaly understanding (VAU) relies on sparse, context-dependent cues.<n>We propose $Anomtext-$, a closed-loop framework that reconceptualizes video understanding as an active sequential decision-making process.<n>Inspired by human video-reviewing behavior, this framework unifies internal cognitive reasoning and strategic evidence acquisition into an interleaved policy.
  arXiv  Detail & Related papers  (2026-07-01T08:41:49Z)
• Provably Secure Agent Guardrail [89.79561918065122]
  Existing defense architectures rely on empirical semantic guardrails and probabilistic large model adjudicators.<n>This paper proposes a new security paradigm for agents based on the fundamental limitations of logical reasoning.
  arXiv  Detail & Related papers  (2026-05-28T02:12:41Z)
• ManipArena: Comprehensive Real-world Evaluation of Reasoning-Oriented Generalist Robot Manipulation [66.02142169323521]
  Vision-Language-Action models and world models have recently emerged as promising paradigms for general-purpose robotic intelligence.<n>Existing benchmarks are largely simulator-centric, which provide controllability but fail to capture the reality gap caused by perception noise.<n>We introduce ManipArena, a standardized evaluation framework designed to bridge simulation and real-world execution.
  arXiv  Detail & Related papers  (2026-03-30T15:06:41Z)
• RailVQA: A Benchmark and Framework for Efficient Interpretable Visual Cognition in Automatic Train Operation [11.03879484190242]
  We introduce RailVQA-bench, the first VQA benchmark for cab-view visual cognition in Automatic Train Operation (ATO)<n>We propose RailVQA-CoM, a collaborative large-small model framework that combines small-model efficiency with large-model cognition.<n> Experiments demonstrate that the proposed approach substantially improves performance, enhances interpretability, reduces inference latency, and strengthens cross-domain generalization.
  arXiv  Detail & Related papers  (2026-03-28T03:41:40Z)
• Agentic Spatio-Temporal Grounding via Collaborative Reasoning [80.83158605034465]
  Temporal Video Grounding aims to retrieve thetemporal tube of a target object or person in a video given a text query.<n>We propose the Agentic Spatio-Temporal Grounder (ASTG) framework for the task of STVG towards an open-world and training-free scenario.<n>Specifically, two specialized agents SRA (Spatial Reasoning Agent) and TRA (Temporal Reasoning Agent) constructed leveraging on modern Multimoal Large Language Models (MLLMs)<n>Experiments on popular benchmarks demonstrate the superiority of the proposed approach where it outperforms existing weakly-supervised and zero-shot approaches by a margin
  arXiv  Detail & Related papers  (2026-02-10T10:16:27Z)
• CogRail: Benchmarking VLMs in Cognitive Intrusion Perception for Intelligent Railway Transportation Systems [29.385460126069386]
  We introduce a novel benchmark, CogRail, which integrates curated datasets with cognitively driven question-answer annotations.<n>Building upon this benchmark, we conduct a systematic evaluation of state-of-the-art visual-language models.<n>We propose a joint fine-tuning framework that integrates three core tasks, position perception, movement prediction, and threat analysis.
  arXiv  Detail & Related papers  (2026-01-14T16:36:26Z)
• Cascading multi-agent anomaly detection in surveillance systems via vision-language models and embedding-based classification [0.0]
  This work introduces a cascading multi-agent framework that unifies complementary paradigms into a coherent and interpretable architecture.<n>Early modules perform reconstruction-gated filtering and object-level assessment, while higher-level reasoning agents are selectively invoked to interpret semantically ambiguous events.<n>The framework advances beyond conventional detection pipelines by combining early-exit efficiency, adaptive multi-agent reasoning, and explainable anomaly attribution, establishing a reproducible and energy-efficient foundation for scalable intelligent visual monitoring.
  arXiv  Detail & Related papers  (2026-01-08T11:31:47Z)
• Robust Verification of Controllers under State Uncertainty via Hamilton-Jacobi Reachability Analysis [49.31947916567367]
  Hamilton-Jacobi (J) reachability analysis is a popular formal verification tool for general nonlinear systems that can compute optimal reachable under worst-case uncertainties.<n>This work is the first HJ-based reachability-based system verification framework for the Robust Verification Controllers via HJ rover.<n>Within Ro-CoRe, we propose novel methods for safety verification and controller design.
  arXiv  Detail & Related papers  (2025-11-18T18:55:20Z)
• OS-Sentinel: Towards Safety-Enhanced Mobile GUI Agents via Hybrid Validation in Realistic Workflows [77.95511352806261]
  Computer-using agents powered by Vision-Language Models (VLMs) have demonstrated human-like capabilities in operating digital environments like mobile platforms.<n>We propose OS-Sentinel, a novel hybrid safety detection framework that combines a Formal Verifier for detecting explicit system-level violations with a Contextual Judge for assessing contextual risks and agent actions.
  arXiv  Detail & Related papers  (2025-10-28T13:22:39Z)
• Verification of Visual Controllers via Compositional Geometric Transformations [49.81690518952909]
  We introduce a novel verification framework for perception-based controllers that can generate outer-approximations of reachable sets.<n>We provide theoretical guarantees on the soundness of our method and demonstrate its effectiveness across benchmark control environments.
  arXiv  Detail & Related papers  (2025-07-06T20:22:58Z)
• Assured Autonomy with Neuro-Symbolic Perception [11.246557832016238]
  Many state-of-the-art AI models deployed in cyber-physical systems (CPS) are pattern-matchers.<n>With limited security guarantees, there are concerns for their reliability in safety-critical and contested domains.<n>We propose a paradigm shift that imbues data-driven perception models with symbolic structure.
  arXiv  Detail & Related papers  (2025-05-27T15:21:06Z)
• T2VShield: Model-Agnostic Jailbreak Defense for Text-to-Video Models [88.63040835652902]
  Text to video models are vulnerable to jailbreak attacks, where specially crafted prompts bypass safety mechanisms and lead to the generation of harmful or unsafe content.<n>We propose T2VShield, a comprehensive and model agnostic defense framework designed to protect text to video models from jailbreak threats.<n>Our method systematically analyzes the input, model, and output stages to identify the limitations of existing defenses.
  arXiv  Detail & Related papers  (2025-04-22T01:18:42Z)
• Unveiling Zero-Space Detection: A Novel Framework for Autonomous Ransomware Identification in High-Velocity Environments [0.0]
  The proposed Zero-Space Detection framework identifies latent behavioral patterns through unsupervised clustering and advanced deep learning techniques.<n>It operates effectively in high-velocity environments by integrating multi-phase filtering and ensemble learning for refined decision-making.<n> Experimental evaluation reveals high detection rates across diverse ransomware families, including LockBit, Conti, REvil, and BlackMatter.
  arXiv  Detail & Related papers  (2025-01-22T11:41:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.