Semantically-Aware Diver Activity Recognition Framework for Effective Underwater Multi-Human-Robot Collaboration

• URL: http://arxiv.org/abs/2606.12374v1
• Date: Wed, 10 Jun 2026 17:40:19 GMT
• Title: Semantically-Aware Diver Activity Recognition Framework for Effective Underwater Multi-Human-Robot Collaboration
• Abstract summary: We introduce DAR-Net, a framework that analyzes complex underwater scenes to classify diver activities.<n>Our contribution lies in a semantically guided learning formulation that couples transformer-based temporal reasoning with pixel-level scene supervision.<n> DAR-Net achieves promising accuracy in recognizing six distinct diver activities, outperforming state-of-the-art models.
• Score: 9.010955965403669
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Effective multi-human-robot collaboration is essential for expanding human-led operations in the challenging and high-risk underwater environment. For autonomous underwater vehicles (AUVs) to become true teammates, they must be able to comprehend their surroundings and recognize a diver's activities to offer assistance and ensure safety. Towards this goal, we introduce DAR-Net, a novel transformer-based framework that analyzes complex underwater scenes to classify diver activities. Our contribution lies in a semantically guided learning formulation that couples transformer-based temporal reasoning with pixel-level scene supervision. This multi-loss training strategy explicitly aligns global activity recognition with local human-robot interaction semantics, which is particularly critical in low-visibility underwater conditions. To address the significant challenge of data scarcity in this domain, we present the first-ever Underwater Diver Activity (UDA) dataset, a foundational resource containing over 2,600 annotated images with pixel-level masks. Through rigorous experimental evaluations in a controlled environment, we demonstrate that DAR-Net achieves promising accuracy in recognizing six distinct diver activities, outperforming state-of-the-art models. While this dataset provides a crucial baseline, our work serves as a pioneering step, laying the groundwork for future research and facilitating the development of more intelligent, collaborative underwater robotic systems.

Related papers

• Digital Twin Supervised Reinforcement Learning Framework for Autonomous Underwater Navigation [0.0]
  This article investigates issues through the case of the BlueROV2, an open platform widely used for scientific experimentation.<n>We propose a deep reinforcement learning approach based on the Proximal Policy Optimization (PPO) algorithm.<n>Results show that the PPO policy consistently outperforms DWA in highly cluttered environments.
  arXiv  Detail & Related papers  (2025-12-11T18:52:42Z)
• IndustryNav: Exploring Spatial Reasoning of Embodied Agents in Dynamic Industrial Navigation [56.43007596544299]
  IndustryNav is the first dynamic industrial navigation benchmark for active spatial reasoning.<n>A study of nine state-of-the-art Visual Large Language Models reveals that closed-source models maintain a consistent advantage.
  arXiv  Detail & Related papers  (2025-11-21T16:48:49Z)
• NAUTILUS: A Large Multimodal Model for Underwater Scene Understanding [60.76337064425815]
  We study the underwater scene understanding methods, which aim to achieve automated underwater exploration.<n>NautData is a dataset containing 1.45 M image-text pairs supporting eight underwater scene understanding tasks.<n>We propose a plug-and-play vision feature enhancement (VFE) module, which explicitly restores clear underwater information.
  arXiv  Detail & Related papers  (2025-10-31T14:00:35Z)
• Learning Underwater Active Perception in Simulation [51.205673783866146]
  Turbidity can jeopardise the whole mission as it may prevent correct visual documentation of the inspected structures.<n>Previous works have introduced methods to adapt to turbidity and backscattering.<n>We propose a simple yet efficient approach to enable high-quality image acquisition of assets in a broad range of water conditions.
  arXiv  Detail & Related papers  (2025-04-23T06:48:38Z)
• Deep Learning-Enhanced Visual Monitoring in Hazardous Underwater Environments with a Swarm of Micro-Robots [8.38975683806005]
  monitoring and exploration of extreme environments, such as underwater storage facilities, is costly, labor-intensive and hazardous.<n>We propose a novel approach that integrates a multi-temporal deep learning network for coordinate prediction, and image reassembly.<n>Results demonstrate very high coordinate prediction accuracy and plausible image assembly, indicating the real-world applicability of our approach.
  arXiv  Detail & Related papers  (2025-03-04T16:19:06Z)
• Sonar-based Deep Learning in Underwater Robotics: Overview, Robustness and Challenges [0.46873264197900916]
  The predominant use of sonar in underwater environments, characterized by limited training data and inherent noise, poses challenges to model robustness.<n>This paper studies sonar-based perception task models, such as classification, object detection, segmentation, and SLAM.<n>It systematizes sonar-based state-of-the-art datasets, simulators, and robustness methods such as neural network verification, out-of-distribution, and adversarial attacks.
  arXiv  Detail & Related papers  (2024-12-16T15:03:08Z)
• CoNav: A Benchmark for Human-Centered Collaborative Navigation [66.6268966718022]
  We propose a collaborative navigation (CoNav) benchmark. Our CoNav tackles the critical challenge of constructing a 3D navigation environment with realistic and diverse human activities. We propose an intention-aware agent for reasoning both long-term and short-term human intention.
  arXiv  Detail & Related papers  (2024-06-04T15:44:25Z)
• Aquatic Navigation: A Challenging Benchmark for Deep Reinforcement Learning [53.3760591018817]
  We propose a new benchmarking environment for aquatic navigation using recent advances in the integration between game engines and Deep Reinforcement Learning. Specifically, we focus on PPO, one of the most widely accepted algorithms, and we propose advanced training techniques. Our empirical evaluation shows that a well-designed combination of these ingredients can achieve promising results.
  arXiv  Detail & Related papers  (2024-05-30T23:20:23Z)
• A Diver Attention Estimation Framework for Effective Underwater Human-Robot Interaction [14.267807345588581]
  Recent advancements in vision-based underwater HRI methods have the capability to interact with their human partners without requiring assistance from a topside operator.<n>In these methods, the AUV assumes that the diver is ready for interaction, while in reality, the diver may be distracted.<n>This paper presents a diver attention estimation framework for AUVs to autonomously determine the attentiveness of a diver.
  arXiv  Detail & Related papers  (2022-09-28T22:08:41Z)
• DARE: AI-based Diver Action Recognition System using Multi-Channel CNNs for AUV Supervision [3.5584173777587935]
  This paper presents DARE, a diver action recognition system that is trained based on Cognitive Autonomous Driving Buddy dataset. DARE is fast and requires only a few milliseconds to classify one stereo-pair, thus making it suitable for real-time underwater implementation.
  arXiv  Detail & Related papers  (2020-11-16T04:05:32Z)
• Batch Exploration with Examples for Scalable Robotic Reinforcement Learning [63.552788688544254]
  Batch Exploration with Examples (BEE) explores relevant regions of the state-space guided by a modest number of human provided images of important states. BEE is able to tackle challenging vision-based manipulation tasks both in simulation and on a real Franka robot.
  arXiv  Detail & Related papers  (2020-10-22T17:49:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.