Fish-Vista: A Multi-Purpose Dataset for Understanding & Identification of Traits from Images
- URL: http://arxiv.org/abs/2407.08027v2
- Date: Thu, 27 Feb 2025 07:06:50 GMT
- Title: Fish-Vista: A Multi-Purpose Dataset for Understanding & Identification of Traits from Images
- Authors: Kazi Sajeed Mehrab, M. Maruf, Arka Daw, Abhilash Neog, Harish Babu Manogaran, Mridul Khurana, Zhenyang Feng, Bahadir Altintas, Yasin Bakis, Elizabeth G Campolongo, Matthew J Thompson, Xiaojun Wang, Hilmar Lapp, Tanya Berger-Wolf, Paula Mabee, Henry Bart, Wei-Lun Chao, Wasila M Dahdul, Anuj Karpatne,
- Abstract summary: We introduce Fish-Visual Trait Analysis (Fish-Vista), the first organismal image dataset designed for the analysis of visual traits of aquatic species directly from images using problem formulations in computer vision.<n>Fish-Vista contains 69,126 annotated images spanning 4,154 fish species, curated and organized to serve three downstream tasks of species classification, trait identification, and trait segmentation.
- Score: 18.79726520825725
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: We introduce Fish-Visual Trait Analysis (Fish-Vista), the first organismal image dataset designed for the analysis of visual traits of aquatic species directly from images using problem formulations in computer vision. Fish-Vista contains 69,126 annotated images spanning 4,154 fish species, curated and organized to serve three downstream tasks of species classification, trait identification, and trait segmentation. Our work makes two key contributions. First, we perform a fully reproducible data processing pipeline to process images sourced from various museum collections. We annotate these images with carefully curated labels from biological databases and manual annotations to create an AI-ready dataset of visual traits, contributing to the advancement of AI in biodiversity science. Second, our proposed downstream tasks offer fertile grounds for novel computer vision research in addressing a variety of challenges such as long-tailed distributions, out-of-distribution generalization, learning with weak labels, explainable AI, and segmenting small objects. We benchmark the performance of several existing methods for our proposed tasks to expose future research opportunities in AI for biodiversity science problems involving visual traits.
Related papers
- FishNet: Deep Neural Networks for Low-Cost Fish Stock Estimation [0.0]
FishNet is an automated computer vision system for both taxonomic classification and fish size estimation.
We use a dataset of 300,000 hand-labeled images containing 1.2M fish of 163 different species.
FishNet achieves a 92% intersection over union on the fish segmentation task, a 89% top-1 classification accuracy on single fish species classification, and a 2.3cm mean absolute error on the fish length estimation task.
arXiv Detail & Related papers (2024-03-16T12:44:08Z) - Deep Neural Network Identification of Limnonectes Species and New Class
Detection Using Image Data [5.943822554753426]
Deep neural networks can successfully automate the classification of an image into a known species group for which it has been trained.
We demonstrate that the algorithm can successfully classify an image into a new class if the image does not belong to the existing classes.
arXiv Detail & Related papers (2023-11-15T02:57:59Z) - UniAP: Towards Universal Animal Perception in Vision via Few-shot
Learning [24.157933537030086]
We introduce UniAP, a novel Universal Animal Perception model that enables cross-species perception among various visual tasks.
By capitalizing on the shared visual characteristics among different animals and tasks, UniAP enables the transfer of knowledge from well-studied species to those with limited labeled data or even unseen species.
arXiv Detail & Related papers (2023-08-19T09:13:46Z) - SimFIR: A Simple Framework for Fisheye Image Rectification with
Self-supervised Representation Learning [105.01294305972037]
We introduce SimFIR, a framework for fisheye image rectification based on self-supervised representation learning.
To learn fine-grained distortion representations, we first split a fisheye image into multiple patches and extract their representations with a Vision Transformer.
The transfer performance on the downstream rectification task is remarkably boosted, which verifies the effectiveness of the learned representations.
arXiv Detail & Related papers (2023-08-17T15:20:17Z) - Discovering Novel Biological Traits From Images Using Phylogeny-Guided
Neural Networks [10.372001949268636]
We present a novel approach for discovering evolutionary traits directly from images without relying on trait labels.
Our proposed approach, Phylo-NN, encodes the image of an organism into a sequence of quantized feature vectors.
We demonstrate the effectiveness of our approach in producing biologically meaningful results in a number of downstream tasks.
arXiv Detail & Related papers (2023-06-05T20:22:05Z) - Domain Generalization for Mammographic Image Analysis with Contrastive
Learning [62.25104935889111]
The training of an efficacious deep learning model requires large data with diverse styles and qualities.
A novel contrastive learning is developed to equip the deep learning models with better style generalization capability.
The proposed method has been evaluated extensively and rigorously with mammograms from various vendor style domains and several public datasets.
arXiv Detail & Related papers (2023-04-20T11:40:21Z) - Learning Transferable Pedestrian Representation from Multimodal
Information Supervision [174.5150760804929]
VAL-PAT is a novel framework that learns transferable representations to enhance various pedestrian analysis tasks with multimodal information.
We first perform pre-training on LUPerson-TA dataset, where each image contains text and attribute annotations.
We then transfer the learned representations to various downstream tasks, including person reID, person attribute recognition and text-based person search.
arXiv Detail & Related papers (2023-04-12T01:20:58Z) - A domain adaptive deep learning solution for scanpath prediction of
paintings [66.46953851227454]
This paper focuses on the eye-movement analysis of viewers during the visual experience of a certain number of paintings.
We introduce a new approach to predicting human visual attention, which impacts several cognitive functions for humans.
The proposed new architecture ingests images and returns scanpaths, a sequence of points featuring a high likelihood of catching viewers' attention.
arXiv Detail & Related papers (2022-09-22T22:27:08Z) - Learning multi-scale functional representations of proteins from
single-cell microscopy data [77.34726150561087]
We show that simple convolutional networks trained on localization classification can learn protein representations that encapsulate diverse functional information.
We also propose a robust evaluation strategy to assess quality of protein representations across different scales of biological function.
arXiv Detail & Related papers (2022-05-24T00:00:07Z) - Series Photo Selection via Multi-view Graph Learning [52.33318426088579]
Series photo selection (SPS) is an important branch of the image aesthetics quality assessment.
We leverage a graph neural network to construct the relationships between multi-view features.
A siamese network is proposed to select the best one from a series of nearly identical photos.
arXiv Detail & Related papers (2022-03-18T04:23:25Z) - Self-Supervised Vision Transformers Learn Visual Concepts in
Histopathology [5.164102666113966]
We conduct a search for good representations in pathology by training a variety of self-supervised models with validation on a variety of weakly-supervised and patch-level tasks.
Our key finding is in discovering that Vision Transformers using DINO-based knowledge distillation are able to learn data-efficient and interpretable features in histology images.
arXiv Detail & Related papers (2022-03-01T16:14:41Z) - A Survey of Fish Tracking Techniques Based on Computer Vision [11.994865945394139]
This paper presents a review of the advancements of fish tracking technologies over the past seven years-2023.
It explores diverse fish tracking techniques with an emphasis on fundamental localization and tracking methods.
It also summarizes open-source datasets, evaluation metrics, challenges, and applications in fish tracking research.
arXiv Detail & Related papers (2021-10-06T07:46:35Z) - Factors of Influence for Transfer Learning across Diverse Appearance
Domains and Task Types [50.1843146606122]
A simple form of transfer learning is common in current state-of-the-art computer vision models.
Previous systematic studies of transfer learning have been limited and the circumstances in which it is expected to work are not fully understood.
In this paper we carry out an extensive experimental exploration of transfer learning across vastly different image domains.
arXiv Detail & Related papers (2021-03-24T16:24:20Z) - Perspectives on individual animal identification from biology and
computer vision [58.81800919492064]
We review current advances of computer vision identification techniques to provide both computer scientists and biologists with an overview of the available tools.
We conclude by offering recommendations for starting an animal identification project, illustrate current limitations and propose how they might be addressed in the future.
arXiv Detail & Related papers (2021-02-28T16:50:09Z) - Deep Low-Shot Learning for Biological Image Classification and
Visualization from Limited Training Samples [52.549928980694695]
In situ hybridization (ISH) gene expression pattern images from the same developmental stage are compared.
labeling training data with precise stages is very time-consuming even for biologists.
We propose a deep two-step low-shot learning framework to accurately classify ISH images using limited training images.
arXiv Detail & Related papers (2020-10-20T06:06:06Z) - Unifying data for fine-grained visual species classification [15.14767769034929]
We present an initial deep convolutional neural network model, trained on 2.9M images across 465 fine-grained species.
The long-term goal is to enable scientists to make conservation recommendations from near real-time analysis of species abundance and population health.
arXiv Detail & Related papers (2020-09-24T01:04:18Z) - A Realistic Fish-Habitat Dataset to Evaluate Algorithms for Underwater
Visual Analysis [2.6476746128312194]
We present DeepFish as a benchmark suite with a large-scale dataset to train and test methods for several computer vision tasks.
The dataset consists of approximately 40 thousand images collected underwater from 20 greenhabitats in the marine-environments of tropical Australia.
Our experiments provide an in-depth analysis of the dataset characteristics, and the performance evaluation of several state-of-the-art approaches.
arXiv Detail & Related papers (2020-08-28T12:20:59Z) - Temperate Fish Detection and Classification: a Deep Learning based
Approach [6.282069822653608]
We propose a two-step deep learning approach for the detection and classification of temperate fishes without pre-filtering.
The first step is to detect each single fish in an image, independent of species and sex.
In the second step, we adopt a Convolutional Neural Network (CNN) with the Squeeze-and-Excitation (SE) architecture for classifying each fish in the image without pre-filtering.
arXiv Detail & Related papers (2020-05-14T12:40:57Z) - Automatic image-based identification and biomass estimation of
invertebrates [70.08255822611812]
Time-consuming sorting and identification of taxa pose strong limitations on how many insect samples can be processed.
We propose to replace the standard manual approach of human expert-based sorting and identification with an automatic image-based technology.
We use state-of-the-art Resnet-50 and InceptionV3 CNNs for the classification task.
arXiv Detail & Related papers (2020-02-05T21:38:57Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.