UniqueShip: Mitigating Data Leakage in Acoustic Ship Classification Benchmark Datasets

• URL: http://arxiv.org/abs/2609.13659v1
• Date: Sat, 12 Sep 2026 02:31:38 GMT
• Title: UniqueShip: Mitigating Data Leakage in Acoustic Ship Classification Benchmark Datasets
• Abstract summary: We introduce UniqueShip, a machine learning-ready benchmark dataset for Underwater Acoustic Target Recognition.<n>Unlike previous datasets, we explicitly control for "data leakage" between the training and evaluation sets.<n>UniqueShip contains 2,460 hours of ship-radiated audio from 4,218 unique vessels.
• Score: 3.8240667211978163
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Underwater Acoustic Target Recognition (UATR) of ships is well-suited for machine learning, yet its progress is hindered by the lack of large, diverse, and publicly available labeled datasets. In this work, we introduce UniqueShip, a machine learning-ready benchmark dataset for UATR applications sourced from the open Ocean Networks Canada (ONC) repository. Unlike previous datasets, we explicitly control for "data leakage" between the training and evaluation sets to ensure more reliable and generalizable model evaluation that does not encourage the model to memorize individual ships. We demonstrate that typical, random data partitioning in two prominent UATR datasets leads to falsely optimistic test performance, increasing accuracy by 10-48 percentage points compared to our more careful partitioning. Ablations on UniqueShip further show that doubling the number of unique vessels improves accuracy by 2.4-2.6 percentage points, while doubling total audio duration improves only by 0.8-1.3 points, indicating that vessel diversity should drive dataset curation more than total hours. We provide baselines with convolutional and transformer backbones, and analyze how ship metadata correlates with classification performance, finding that individual vessel characteristics predict classification difficulty far better than distance to the hydrophone alone. Overall, UniqueShip contains 2,460 hours of ship-radiated audio from 4,218 unique vessels (3,437 hours including background). We publish the dataset, code, and easy-to-download splits at uniqueshipdata.org to foster further UATR research.

Related papers

• WUTDet: A 100K-Scale Ship Detection Dataset and Benchmarks with Dense Small Objects [68.44670476527155]
  WUTDet is a large-scale ship detection dataset with 100,576 images and 381,378 annotated ship instances.<n>We systematically evaluate 20 baseline models from three mainstream detection architectures, namely CNN, Transformer, and Mamba.<n>Results on Ship-GEN show that models trained on WUTDet exhibit stronger generalization under different data distributions.
  arXiv  Detail & Related papers  (2026-04-09T03:28:05Z)
• NASTaR: NovaSAR Automated Ship Target Recognition Dataset [1.8957631297817699]
  This dataset comprises of 3415 ship patches extracted from NovaSAR S-band imagery.<n>It includes distinctive features such as 23 unique classes, inshore/offshore separation, and an auxiliary wake dataset for patches where ship wakes are visible.<n>We validated the dataset applicability across prominent ship-type classification scenarios using benchmark deep learning models.
  arXiv  Detail & Related papers  (2025-12-20T20:42:30Z)
• OpenDataArena: A Fair and Open Arena for Benchmarking Post-Training Dataset Value [74.80873109856563]
  OpenDataArena (ODA) is a holistic and open platform designed to benchmark the intrinsic value of post-training data.<n>ODA establishes a comprehensive ecosystem comprising four key pillars: (i) a unified training-evaluation pipeline that ensures fair, open comparisons across diverse models; (ii) a multi-dimensional scoring framework that profiles data quality along tens of distinct axes; and (iii) an interactive data lineage explorer to visualize dataset genealogy and dissect component sources.
  arXiv  Detail & Related papers  (2025-12-16T03:33:24Z)
• Machine Learning-Based Classification of Vessel Types in Straits Using AIS Tracks [0.0]
  This paper presents a machine-learning pipeline that classifies moving vessels using only AIS data.<n>We analyze eight days of historical AIS from the Danish Maritime Authority covering the Bornholm Strait in the Baltic Sea.
  arXiv  Detail & Related papers  (2025-09-09T19:52:43Z)
• Mixed Signals: A Diverse Point Cloud Dataset for Heterogeneous LiDAR V2X Collaboration [57.3519952529079]
  Vehicle-to-everything (V2X) collaborative perception has emerged as a promising solution to address the limitations of single-vehicle perception systems.<n>To address these gaps, we present Mixed Signals, a comprehensive V2X dataset featuring 45.1k point clouds and 240.6k bounding boxes.<n>Our dataset provides point clouds and bounding box annotations across 10 classes, ensuring reliable data for perception training.
  arXiv  Detail & Related papers  (2025-02-19T23:53:00Z)
• Prediction of Vessel Arrival Time to Pilotage Area Using Multi-Data Fusion and Deep Learning [9.458664533786994]
  This paper investigates the prediction of vessels' arrival time to the pilotage area using multi-data fusion and deep learning approaches. Tests on two real-world data sets from Singapore have been conducted and the following promising results have been obtained.
  arXiv  Detail & Related papers  (2024-03-15T02:25:04Z)
• Collaborative Learning with a Drone Orchestrator [79.75113006257872]
  A swarm of intelligent wireless devices train a shared neural network model with the help of a drone. The proposed framework achieves a significant speedup in training, leading to an average 24% and 87% saving in the drone hovering time.
  arXiv  Detail & Related papers  (2023-03-03T23:46:25Z)
• Navya3DSeg -- Navya 3D Semantic Segmentation Dataset & split generation for autonomous vehicles [63.20765930558542]
  3D semantic data are useful for core perception tasks such as obstacle detection and ego-vehicle localization. We propose a new dataset, Navya 3D (Navya3DSeg), with a diverse label space corresponding to a large scale production grade operational domain. It contains 23 labeled sequences and 25 supplementary sequences without labels, designed to explore self-supervised and semi-supervised semantic segmentation benchmarks on point clouds.
  arXiv  Detail & Related papers  (2023-02-16T13:41:19Z)
• ABOShips -- An Inshore and Offshore Maritime Vessel Detection Dataset with Precise Annotations [0.17205106391379021]
  Maritime vessel detection of inshore and offshore datasets is no exception. We collected a dataset of images of maritime vessels taking into account different factors. Vessel instances (including 9 types of vessels), seamarks and miscellaneous floaters were precisely annotated. We evaluated the the out-of-the-box performance of four prevalent object detection algorithms.
  arXiv  Detail & Related papers  (2021-02-11T07:05:33Z)
• Real-Time target detection in maritime scenarios based on YOLOv3 model [65.35132992156942]
  A novel ships dataset is proposed consisting of more than 56k images of marine vessels collected by means of web-scraping. A YOLOv3 single-stage detector based on Keras API is built on top of this dataset.
  arXiv  Detail & Related papers  (2020-02-10T15:25:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.