Context-Driven Detection of Invertebrate Species in Deep-Sea Video
- URL: http://arxiv.org/abs/2206.00718v1
- Date: Wed, 1 Jun 2022 18:59:46 GMT
- Title: Context-Driven Detection of Invertebrate Species in Deep-Sea Video
- Authors: R. Austin McEver, Bowen Zhang, Connor Levenson, A S M Iftekhar, B.S.
Manjunath
- Abstract summary: We present a benchmark suite to train, validate, and test methods for temporally localizing four underwater substrates and 59 underwater invertebrate species.
DUSIA currently includes over ten hours of footage across 25 videos captured in 1080p at 30 fps by an ROV.
Some frames are annotated with precise bounding box locations for invertebrate species of interest.
- Score: 11.38215488702246
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Each year, underwater remotely operated vehicles (ROVs) collect thousands of
hours of video of unexplored ocean habitats revealing a plethora of information
regarding biodiversity on Earth. However, fully utilizing this information
remains a challenge as proper annotations and analysis require trained
scientists time, which is both limited and costly. To this end, we present a
Dataset for Underwater Substrate and Invertebrate Analysis (DUSIA), a benchmark
suite and growing large-scale dataset to train, validate, and test methods for
temporally localizing four underwater substrates as well as temporally and
spatially localizing 59 underwater invertebrate species. DUSIA currently
includes over ten hours of footage across 25 videos captured in 1080p at 30 fps
by an ROV following pre planned transects across the ocean floor near the
Channel Islands of California. Each video includes annotations indicating the
start and end times of substrates across the video in addition to counts of
species of interest. Some frames are annotated with precise bounding box
locations for invertebrate species of interest, as seen in Figure 1. To our
knowledge, DUSIA is the first dataset of its kind for deep sea exploration,
with video from a moving camera, that includes substrate annotations and
invertebrate species that are present at significant depths where sunlight does
not penetrate. Additionally, we present the novel context-driven object
detector (CDD) where we use explicit substrate classification to influence an
object detection network to simultaneously predict a substrate and species
class influenced by that substrate. We also present a method for improving
training on partially annotated bounding box frames. Finally, we offer a
baseline method for automating the counting of invertebrate species of
interest.
Related papers
- Multiview Aerial Visual Recognition (MAVREC): Can Multi-view Improve
Aerial Visual Perception? [57.77643186237265]
We present Multiview Aerial Visual RECognition or MAVREC, a video dataset where we record synchronized scenes from different perspectives.
MAVREC consists of around 2.5 hours of industry-standard 2.7K resolution video sequences, more than 0.5 million frames, and 1.1 million annotated bounding boxes.
This makes MAVREC the largest ground and aerial-view dataset, and the fourth largest among all drone-based datasets.
arXiv Detail & Related papers (2023-12-07T18:59:14Z) - Multi-label Video Classification for Underwater Ship Inspection [2.537406035246369]
We propose an automatic video analysis system using deep learning and computer vision to improve upon existing methods.
Our proposed method has demonstrated promising results and can serve as a benchmark for future research and development in underwater video hull inspection applications.
arXiv Detail & Related papers (2023-05-27T02:38:54Z) - A Dataset with Multibeam Forward-Looking Sonar for Underwater Object
Detection [0.0]
Multibeam forward-looking sonar (MFLS) plays an important role in underwater detection.
There are several challenges to the research on underwater object detection with MFLS.
We present a novel dataset, consisting of over 9000 MFLS images captured using Tritech Gemini 1200ik sonar.
arXiv Detail & Related papers (2022-12-01T08:26:03Z) - TempNet: Temporal Attention Towards the Detection of Animal Behaviour in
Videos [63.85815474157357]
We propose an efficient computer vision- and deep learning-based method for the detection of biological behaviours in videos.
TempNet uses an encoder bridge and residual blocks to maintain model performance with a two-staged, spatial, then temporal, encoder.
We demonstrate its application to the detection of sablefish (Anoplopoma fimbria) startle events.
arXiv Detail & Related papers (2022-11-17T23:55:12Z) - Video Salient Object Detection via Contrastive Features and Attention
Modules [106.33219760012048]
We propose a network with attention modules to learn contrastive features for video salient object detection.
A co-attention formulation is utilized to combine the low-level and high-level features.
We show that the proposed method requires less computation, and performs favorably against the state-of-the-art approaches.
arXiv Detail & Related papers (2021-11-03T17:40:32Z) - FathomNet: A global underwater image training set for enabling
artificial intelligence in the ocean [0.0]
Ocean-going platforms are integrating high-resolution camera feeds for observation and navigation, producing a deluge of visual data.
Recent advances in machine learning enable fast, sophisticated analysis of visual data, but have had limited success in the oceanographic world.
We will demonstrate how machine learning models trained on FathomNet data can be applied across different institutional video data.
arXiv Detail & Related papers (2021-09-29T18:08:42Z) - ASOD60K: Audio-Induced Salient Object Detection in Panoramic Videos [79.05486554647918]
We propose PV-SOD, a new task that aims to segment salient objects from panoramic videos.
In contrast to existing fixation-level or object-level saliency detection tasks, we focus on multi-modal salient object detection (SOD)
We collect the first large-scale dataset, named ASOD60K, which contains 4K-resolution video frames annotated with a six-level hierarchy.
arXiv Detail & Related papers (2021-07-24T15:14:20Z) - Detection of Deepfake Videos Using Long Distance Attention [73.6659488380372]
Most existing detection methods treat the problem as a vanilla binary classification problem.
In this paper, the problem is treated as a special fine-grained classification problem since the differences between fake and real faces are very subtle.
A spatial-temporal model is proposed which has two components for capturing spatial and temporal forgery traces in global perspective.
arXiv Detail & Related papers (2021-06-24T08:33:32Z) - AcinoSet: A 3D Pose Estimation Dataset and Baseline Models for Cheetahs
in the Wild [51.35013619649463]
We present an extensive dataset of free-running cheetahs in the wild, called AcinoSet.
The dataset contains 119,490 frames of multi-view synchronized high-speed video footage, camera calibration files and 7,588 human-annotated frames.
The resulting 3D trajectories, human-checked 3D ground truth, and an interactive tool to inspect the data is also provided.
arXiv Detail & Related papers (2021-03-24T15:54:11Z) - A Realistic Fish-Habitat Dataset to Evaluate Algorithms for Underwater
Visual Analysis [2.6476746128312194]
We present DeepFish as a benchmark suite with a large-scale dataset to train and test methods for several computer vision tasks.
The dataset consists of approximately 40 thousand images collected underwater from 20 greenhabitats in the marine-environments of tropical Australia.
Our experiments provide an in-depth analysis of the dataset characteristics, and the performance evaluation of several state-of-the-art approaches.
arXiv Detail & Related papers (2020-08-28T12:20:59Z) - FathomNet: An underwater image training database for ocean exploration
and discovery [0.0]
FathomNet is a novel baseline image training set optimized to accelerate development of modern, intelligent, and automated analysis of underwater imagery.
To date, there are more than 80,000 images and 106,000 localizations for 233 different classes, including midwater and benthic organisms.
While we find quality results on prediction for this new dataset, our results indicate that we are ultimately in need of a larger data set for ocean exploration.
arXiv Detail & Related papers (2020-06-30T21:23:06Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.